Venue: NeurIPS 2021 Conference

Reviews:

final_score: rating: 6: Marginally above the acceptance threshold
review_content: summary: The authors present a new neural network system that can reason over entities. The solution should be applicable to any tasks that require visual reasoning.

Specifically, the NPS algorithm is:
```
for every step in a sequence:
    update the slots (entities)
    compute attention between slot queries and rule keys to select a rule
    compute attention between primary slot queries and all slot keys to get contextual slot
    apply the rule MLP to the primary and contextual slot values
    add the result to the primary slot
```
Slots can either be updated in parallel (i.e., there is only one slot updated per step) or in sequence (i.e., multiple slots can be updated per step).

They compare sequential vs parallel rule application on 4 tasks:
- given a transformation and mnist digit, predict the digit with the transformation applied
- given two x,y coordinates predict the result of an arithmetic change (e.g., subtract the ys from one another) applied
- for a stack of falling shapes, predict the bounding box of each object in future frames
- predict the next frame of billiard balls bouncing around

They compare NPS to GNNs and related methods on four tasks. The goal of each task is to predict latent states in future frames:
- physics env: blocks of unknown weight interacting with one another
- Atari games (pong, space invaders, freeway, breakout, qBert)
- sprites (moving objects of different shapes)
main_review: Originality: NPS is a novel system for visual reasoning. The computational components borrow heavily from attention mechanisms but separately represent rules to apply.

Quality: The authors test a large number of total tasks, targeting different aspects of NPS. E.g., using the first set of experimental results they are able to provide the recommendation that sequential application is better for sparse interactions.

Clarity: I would have appreciated a more straightforward introduction that was heavier on visuals. E.g., the paragraph beginning at section 64 discusses going from raw observations to slots, which is not what NPS does. It takes up a lot of real estate in the introduction and gave me the impression that the paper focused on going from raw data to slots. It's accompanied by Figure 1, which provides a visualization of rules before they're introduced, which I found confusing. I think it would help with clarity to add more visualizations of the NPS system into the main text (or to at least pull the algorithm into the main text). I also raise some minor inconsistencies that caused confusion under minor comments.

Significance: The potential impact for differentiable reasoning is enormous. The aim of this paper is to take a step towards goal by showing a reasoning system on a variety of restricted settings that are straightforward to analyze. On many tasks, NPS is compared to only one baseline and provides a small performance improvement. E.g., on shapes stack, bouncing balls (test), and sprites, NPS is within 1-2 points of the baseline (and well within the error bars). On physics env and Atari, NPS has a more sizable gain, but is still within the error bars.

Minor comments:
- Algorithm 1 should specify that j indexes M (num slots)
- The steps in algorithm 1 should be consistent with main text (it omits the primary slot selection)
- L208: an -> a?
- L402 figure cuts off the text
- "Entity abstraction in visual model-based reinforcement learning." is cited twice?

Questions to authors:
- how would you contrast NPS to a transformer trained on the same tasks, but with a rule "memory store"?

Related work of interest:
- Visual Grounding of Learned Physical Models. Li et. al, 2020.
- Learning visual predictive models of physics for playing billiards. Fragkiadaki et. al., 2017.
limitations_and_societal_impact: The authors addressed limitation and impact.
review_title: null

final_score: rating: 8: Top 50% of accepted NeurIPS papers, clear accept
review_content: summary: The paper presents Neural Production Systems (NPS), a neural model to build entiity-centric representations and model the latent rules that determine their interactions in visual environments, inspired by production systems.
main_review: Strengths:
- I found this paper very interesting. Although inspired by an old concept (i.e., production systems), its deep learning adaptation is far from trivial.
- The reported experiments are convincing. In particular, the authors show that NPS can learn arithmetic rules, map rules to MNIST transformation, work in richer visual settings and perform future-state predictions (such as predicting the number of future steps in action-conditioned world models) better than GNNs.
- Rules are differentiable, and could be learned in the context of a broader framework.

Questions:
- How does the model scale with respect to the number of entities/rules considered?

Presentation comments:
- Figure 1 is not colour-blind friendly.
- It might be useful to report results for individual games in Table 5, with the number of entities/rules considered.
limitations_and_societal_impact: the authors adequately addressed the limitations and potential negative societal impact of their work
review_title: null

final_score: rating: 7: Good paper, accept
review_content: summary: This paper proposes a novel method that combines some of the strengths of deep learning with *production systems*, a classic approach in symbolic AI and cognitive science, and evaluates the method on physical reasoning tasks.
main_review: The idea is very exciting and I think it has a lot of promise. Furthermore, the algorithms proposed in the paper creatively combine a range of deep learning techniques (slot-based object-centric representations, key/value attention, gumbel softmax) in an attempt to accomplish this unification. Unfortunately, I think in its current state this work suffers from two limitations:

1) The results: 
* The current set of experiments seem to demonstrate only marginal improvements over the baselines in some cases. In many instances there is no specification of how many random seeds are being averaged over, or the number of seeds is very small, and it is not clear whether there is even a statistically significant advantage over the baseline methods. 
* Furthermore there is typically only one baseline evaluated for each experiment, and it is not clearly specified why this baseline was chosen or whether it represents the current state-of-the-art for the particular metric being evaluated. This is not the only important consideration, but it is certainly important information to include.

2. The algorithm(s): There are significant questions about the generality of the proposed algorithm, at least in its current formulation. 
* First, the algorithm is applied somewhat inconsistently to the different experiments, in a way that isn’t entirely transparent. For instance, in the MNIST transformation task, it sounds like the task instruction (specifying the transformation to be performed) is used to query the rules, which, according to their formulation, should make the task instruction itself the ‘primary slot’, meaning that it should be updated by the selected rule. But presumably it is actually the image embedding that is treated as the primary slot, so as to apply the selected transformation. This experiment is only intended as a preliminary test of the model, but I think it actually reveals a way in which important aspects of the implementation have to be manually tailored to a particular task, because the rule selection and application needs to be handled in an entirely different way if rules are selected on the basis of an instruction vs. on the basis of the objects themselves. Another example comes from the coordinate arithmetic task, in which the coordinates need to be assigned to slots in a particular way in order for the model to successfully use the proposed rule selection method. Specifically, the input and output coordinates for each dimension need to be concatenated into a single slot in order to allow the rule to be inferred from pairwise comparisons between each slot and each rule embedding. If, instead, the input and output coordinates were each represented as separate slots (which seems like a natural way to apply the algorithm as currently specified) there would be no way to identify the rule governing the transformation from input to output.
* Additionally, there are two versions of the proposed method, sequential vs. parallel NPS, and which version works better is somewhat task-dependent. The authors offer reasonable conjectures as to why a particular method works better on a particular task, but this task-dependency does somewhat undermine their claim that sparse interactions are a generally useful inductive bias. The parallel version, which is somewhat less sparse than the sequential version, works better on tasks involving multiple closely interacting objects, whereas the more sparse sequential version works better on tasks involving more spatially distributed objects that mostly don’t interact with each other. But presumably the task of real-world physical reasoning will involve a mixture of these scenarios, and there is no proposal for how this decision should be made other than by the intuitive judgment of the modeler.
* Finally, the current formulation seems to be overly tailored to physical reasoning tasks, and lacking the more general expressive power of the original production systems. For instance, the current formulation only accommodates binary interactions, and there is no way to achieve any kind of hierarchy (i.e. in which the output of one rule serves as the input to another). Many of the design decisions regarding how rules are selected and applied seem as though they have been made specifically with physical reasoning in mind, and it’s not clear how they will be modified to handle other sorts of tasks to which production systems have traditionally been applied, especially tasks involving more abstract reasoning. 

In summary, I think the current approach might be more conservatively described as a method for efficiently modeling sparse interactions for the purposes of physical reasoning, rather than as a ‘neural production system’ which seems to imply much more general functionality than it currently has. Nevertheless, I think that it might still merit acceptance if the issues regarding the results and baselines are resolved.

Minor points:
* Some of the notation is inconsistent or unclear. For instance, in the description of step 2 (line 149), the key is obtained using a variable denoted as $R_i$, is this the same as the learned rule embedding vector (described in line 124) denoted as $\vec{\textbf{R}}_i$? Is $\textbf{k}_i$ simply equal to $\vec{\textbf{R}}_i$ as indicated in algorithms 1 and 2, or is it multiplied by the weights $W^k$ as indicated on line 149? In Algorithm 2, are the weights $W^k$ really used to obtain $\textbf{q}_p$ or are these supposed to be $W^q$ or $\widetilde{W}^q$? Also in Algorithm 2, is $\hat{W}^k$ an additional set of weights not specified in the input description, or is it supposed to be $W^k$?
* In the Appendix it says that Table 3 in the paper is wrong and that Table 9 (in the Appendix) reflects the correct results. Hopefully Table 9 will go into the main body of the paper, including the results for both the test and transfer regimes?
* It would be helpful to include pointers to the specific relevant sections in the Appendix, and to try and arrange the sections of the Appendix in the same order as the corresponding sections in the main body of the paper.

**Update after discussion period**

The authors have made improvements to the algorithm, clarified the criteria for selecting baselines and how these relate to the current state-of-the-art, and performed more extensive experiments which demonstrate that the improved algorithm clearly outperforms other approaches on a range of tasks. Notably, these involved experiments on a Raven's Progressive Matrices benchmark, showing some improvements in out-of-distribution generalization, suggesting that NPS may useful in more symbolic domains in addition to the physical reasoning tasks that are the focus of most of the paper. I believe the paper merits acceptance and am updating my score to a 7.

limitations_and_societal_impact: Yes, the authors have adequately addressed the limitations and potential negative societal impact of the work.
review_title: null

final_score: rating: 6: Marginally above the acceptance threshold
review_content: summary: This paper proposes Neural Production Systems (NPS) which is a (neural) rule-based learning system that operate on explicit entities. The main idea is to extend traditional Production systems to be end-to-end learnable with neural networks. Unlike previous NN systems that model entity-wise interactions such as GNN, NPS allows sparse interactions between entities which is favorable in certain scenarios. 
main_review: Originality & Significance
- The idea of extending the traditional rule-based system to neural net rule-based system is intriguing as it can combine the benefit of excellent perception of NN and generalization of rule-based systems. Also, rules in a given environment can be naturally discovered through the learning process. This paper shows promising results on how rules can be learned. However, I have a concern that this paper seems like a marginal improvement over the previously proposed SCOFF (Goyal et al. 2020) model which I further discuss below. 

Quality
- SCOFF model also has soft-attention based selection process for entities and hard selection with Gumbel-softmax for schema. Although the intuition and exact formulation is different, the rule application process in NPS can be modelled using schema+soft competition among entities in SCOFF. In that case, the main benefit of NPS comes from having the sparse interaction between entities because it can be more efficient and easier to learn which could be valuable if there is a significant improvement in performance. Therefore, the experiments section should demonstrate that better in my opinion. 
- Exp could include more analysis on how NPS can learn certain rules that SCOFF cannot. It would be great to see if the improvements in NPS is enabling the perfect separation of rules in Sec4.1 and if SCOFF cannot do it perfectly. 
- The shapes stack and Sec4.3 could also compare with SCOFF as a stronger baseline.
- The internal behavior of NPS is analyzed for simple tasks such as MNIST transformation, but systems like NPS can really shine when applied to complex environments with many entities such as Bouncing Balls. The results would be strengthend if there are detailed analysis or visualization of how entities are selected and how rules are applied to the selected entities.
- Line 402 and Figure 5 are overlapping.

Clarity
- The paper is easy to read and well-written.
limitations_and_societal_impact: Yes.
review_title: null

Meta-review: This paper introduces a differentiable and learnable version of the classic production system architecture. This is a worthwhile and interesting attempt at enabling more flexible reasoning in deep learning systems. The reviewers found the initial evaluations of the system uncompelling, however the substantial additions provided during the discussion phase were deemed sufficient by all reviewers. This means the paper is an unusual case where it will need substantial revision to the results, but the reviewers believe this is feasible -- so I too will recommend acceptance. I urge the authors to also attempt to clarify the model presentation, as indicated by the reviewers.
Decision: Accept (Poster)