Venue: ICLR 2018 conference

Reviews:

final_score: rating: 6: Marginally above acceptance threshold
review_content: I thank the authors for their updates and clarifications.  I stand by my original review and score.  I think their method and their evaluation has some major weaknesses, but I think that it still provides a good baseline to force work in this space towards tasks which can not be solved by simpler models like this.  So while I'm not super excited about the paper I think it is above the accept threshold.
--------------------------------------------------------------------------
This paper extends an existing thread of neural computation research focused on learning resuable subprocedures (or options in RL-speak).  Instead of simply input and output examples, as in most of the work in neural computation, they follow in the vein of the Neural Programmer-Interpreter (Reed and de Freitas, 2016) and Li et. al., 2017, where the supervision contains the full sequence of elementary actions in the domain for all samples, and some samples also contain the hierarchy of subprocedure calls.

The main focus of their work is learning from fewer fully annotated samples than prior work.  They introduce two new ideas in order to enable this:
1.  They limit the memory state of each level in the program heirarchy to simply a counter indicating the number of elementary actions/subprocedure calls taken so far (rather than a full RNN embedded hidden/cell state as in prior work).  They also limit the subprocedures such that they do not accept any arguments.
2.  By considering this very limited set of possible hidden states, they can compute the gradients using a dynamic program that seems to be more accurate than the approximate dynamic program used in Li et. al., 2017.  

The main limitation of the work is this extremely limited memory state, and the lack of arguments.  Without arguments, everything that parameterizes the subprocedures must be in the visible world state.  In both of their domains, this is true, but this places a significant limitation on the algorithms which can be modeled with this technique.  Furthermore, the limited memory state means that the only way a subprocedure can remember anything about the current observation is to call a different subprocedure.  Again, their two evalation tasks fit into this paradigm, but this places very significant limitations on the set of applicable domains.  I would have like to see more discussion on how constraining these limitations would be in practice.  For example, it seems it would be impossible for this architecture to perform the Nanocraft task if the parameters of the task (width, height, etc.) were only provided in the first observation, rather than every observation.  

None-the-less I think this work is an important step in our understanding of the learning dynamics for neural programs.  In particular, while the RNN hidden state memory used by the prior work enables the learning of more complicted programs *in theory*, this has not been shown in practice. So, it's possible that all the prior work is doing is learning to approixmate a much simpler architecture of this form.  Specifically, I think this work can act as a great base-line by forcing future work to focus on domains which cannot be easily solved by a simpler architecture of this form.  This limitation will also force the community to think about which tasks require a more complicated form of memory, and which can be solved with a very simple memory of this form.


I also have the following additional concerns about the paper:

1.  I found the current explanation of the algorithm to be very difficult to understand.  It's extremely difficult to understand the core method without reading the appendix, and even with the appendix I found the explanation of the level-by-level decomposition to be too terse.

2.  It's not clear how their gradient approximation compares to the technique used by Li et. al.  They obviously get better results on the addition and Nanocraft domains, but I would have liked a more clear explanation and/or some experiments providing insights into what enables these improvements (or at least an admission by the authors that they don't really understand what enabled the performance improvements).

review_title: NPI with less supervision

final_score: rating: 6: Marginally above acceptance threshold
review_content: Summary of paper: The goal of this paper is to be able to construct programs given data consisting of program input and program output pairs. Previous works by Reed & Freitas (2016) (using the paper's references) and Cai et al. (2017) used fully supervised trace data. Li et al. (2017) used a mixture of fully supervised and weakly supervised trace data. The supervision helps with discovering the hierarchical structure in the program which helps generalization to other program inputs. The method is heavily based on the "Discovery of Deep Options" (DDO) algorithm by Fox et al. (2017).

---

Quality: The experiments are chosen to compare the method that the paper is proposing directly with the method from Li et al. (2017).
Clarity: The connection between learning a POMDP policy and program induction could be made more explicit. In particular, section 3 describes the problem statement but in terms of learning a POMDP policy. The only sentence with some connection to learning programs is the first one.
Originality: This line of work is very recent (as far as I know), with Li et al. (2017) being the other work tackling program learning from a mixture of supervised and weakly supervised program trace data.
Significance: The problem that the paper is solving is significant. The paper makes good progress in demonstrating this on toy tasks.

---

Some questions/comments:
- Is the Expectation-Gradient trick also known as the reinforce/score function trick?
- This paper could benefit from being rewritten so that it is in one language instead of mixing POMDP language used by Fox et al. (2017) and program learning language. It is not exactly clear, for example, how are memory states m_t and states s_t related to the program traces.
- It would be nice if the experiments in Figure 2 could compare PHP and NPL on exactly the same total number of demonstrations.

---

Summary: The problem under consideration is important and experiments suggest good progress. However, the clarity of the paper could be made better by making the connection between POMDPs and program learning more explicit or if the algorithm was introduced with one language.
review_title: .

final_score: rating: 6: Marginally above acceptance threshold
review_content: In the paper titled "Parameterized Hierarchical Procedures for Neural Programming", the authors proposed "Parametrized Hierarchical Procedure (PHP)", which is a representation of a hierarchical procedure by differentiable parametrization. Each PHP is represented with two multi-layer perceptrons with ReLU activation, one for its operation statement and one for its termination statement. With two benchmark tasks (NanoCraft and long-hand addition), the authors demonstrated that PHPs are able to learn neural programs accurately from smaller amounts of strong/weak supervision. 

Overall the paper is well-written with clear logic and accurate narratives. The methodology within the paper appears to be reasonable to me. Because this is not my research area, I cannot judge its technical contribution. 
review_title: a good paper

Meta-review: This paper is somewhat incremental on recent prior work in a hot area; it has some weaknesses but does move the needle somewhat on these problems.
Decision: Accept (Poster)