Venue: ICLR 2023 Conference

Reviews:

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: This paper is about the task of object rearrangement. The paper proposes a neurosymbolic approach for solving this problem.
The method is described as a hierarchical abstraction approach-- object representations are learned, and a factorized transition graph is constructed over these representations.  The method, "Neural Constraint Satisfaction" disentangles representation learning from planning and control, therefore leading to a modularized approach.  Experiments are performed on three block-style datasets.

strength_and_weaknesses: *Strengths*
- Object rearrangement using image inputs is a very interesting problem and I can see a lot of domains where it could be useful: robotics, planning, visual reasoning, navigation / RL to name a few.
- The paper is well written and does a good job of establishing the preliminaries and the method (algo 1, 2, and the figures are especially useful in this regard)
- experiments appear to be extensive on three datasets/environments (although, note that I am not an expert in RL/planning), with recent baselines
- The neuro-symbolic system proposed has the potential to be applicable in various domains, by simply swapping the visual module.

*Weaknesses*
- One weakness (which is fixable) is that IMO it would be better to introduce the problem statement very early on so that readers get a taste of what's to come later.  The introduction is dedicated to mostly explaining the importance and difficulty of object rearrangement, without specifying what input-output spaces are being considered -- this comes very late.
- It would be interesting to see the method being applied to real world datasets -- i.e. images with real objects (see my suggestion 2 for prospective datasets). Perhaps in such settings, more sophisticated feature learning / entity abstraction methods would need to be used.
- ablation study/analyses to study each module could enhance the evidence for the efficacy of the method

*Suggestions*
1. Increase font size in all figures. Tables are fine.
2. References: there are two related papers that fall under the problem formulation given in Sec 2 Page 2:
  - https://openaccess.thecvf.com/content_CVPRW_2019/papers/Vision_Meets_Cognition_Camera_Ready/Gokhale_Cooking_With_Blocks__A_Recipe_for_Visual_Reasoning_on_CVPRW_2019_paper.pdf This paper is about an object rearrangement task in the "blocksworld" setting but with real images. This one seems to be very similar to _"block-stack"_ from this paper.
  - https://www.ecva.net/papers/eccv_2020/papers_ECCV/papers/123560324.pdf procedural planning in videos -- this paper involves complex actions beyond simple rearrangement, but in real-world videos like cooking etc.

clarity,_quality,_novelty_and_reproducibility: Clarity: well-written paper. Figures can be improved.
Quality: good. analysis from appendix can be moved to the main paper.
Novelty: Visual planning (called object rearrangement in this paper) is actually a very hard task.  I expect the neurosymbolic method introduced in this paper to have a decent impact on the vision+reasoning domain.
Reproducibility: The algorithms are implementation description are useful, but I didn't see any uploaded code.
summary_of_the_review: An effective neurosymbolic method for object re-arrangement, effective across many domains.  Real-world applicability (i.e. with complex images) has not been explored -- all the experimental settings are in synthetic environments, which in my opinion, limits the impact of this paper
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: This paper aims to tackle the combinatorial generalization problem in the offline sensorimotor object rearrangement task. By homogenizing the dynamics of different objects, the authors propose a framework named NCS that trains the correspondence matching module and plans a path from the initial state to the goal state in a non-parametric manner. Results on the proposed environments demonstrate that the proposed method significantly outperforms the offline RL baselines.
strength_and_weaknesses: Strengths:
1. This paper studies an important problem, combinatorial generalization.
2. The authors aim to improve the scalability issue of the previous neural symbolic approaches.

Weaknesses:
1. This paper makes strong assumptions that limit their scope. For example, in the heterogeneous scenario where objects are significantly different in scale (e.g. side length), we cannot directly transfer the knowledge (e.g., offline transition) of a large object to a smaller one.
2. The planning and control algorithms are not clear. For instance, how to search for a reasonable plan without conflicts. I also cannot figure out how to align the entities concretely from only one sentence minimizes the matching cost between z_t and z_g.'. I believe this section is more important than the training details of the submodules. Thus, I recommend that the authors re-organize the paper to emphasize this section more.
3. There are many typos in this paper. I recommend that authors improve their writing, especially in the method section.
clarity,_quality,_novelty_and_reproducibility: I believe this paper has large room to improve writing:
1. Informal illustrations. The captions in Figure 1~5 are hard to read. Figures 5, 7, and 8 are not in pdf format. Besides, it is interesting to see four tables in 'Figure 6'.
2. There are many typos in the paper.
3. Section 3.2 is essential, yet it is not clear enough.
summary_of_the_review: Overall, the authors touch on an important problem in machine learning. Improving the scalability of the non-parametric methods is also appreciated. However, the assumptions made in the paper are strong and there is much room to improve the writing of this paper.
So I lean towards rejecting this paper.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: This paper studies the problem of object rearrangement and focuses particularly on the design of an object-centric planner capable of solving rearrangement tasks requiring multi-step reasoning and systematic generalization. A method, NCS, for combining object-centric representation learning with non-parametric planning is introduced. NCS uses a two-level abstract hierarchy for planning, where a set-structured first level corresponds to sets of entities, and the graph-structured second level corresponds to transitions between “abstracted” entities. Experiments on a few synthetic object rearrangement tasks demonstrate promising performance compared to offline MBRL and object-centric shooting-based planners.

strength_and_weaknesses: Strengths
=======
The main high-level idea, which I take to be the proposition of building a higher level of abstraction on top of object-centric representations upon which it is easier to conduct long-term planning --- is solid. NCS appears to be a promising first step based on the decent empirical results on synthetic tasks.

Weaknesses
=======
Given that the key contribution of this paper appears, to me, to be the NCS planner as a whole (specifically, as a demonstration of how object-centric representations can be leveraged for object rearrangement tasks), NCS’s design appears to rely too heavily on certain strong assumptions that cater to the synthetic rearrangement environments it is demonstrated on. I believe this makes it difficult to see how NCS’s technical designs can inform future frameworks applied on more challenging environments. In particular, the treatment of object identity is highly simplistic; NCS copies the identity of each object discovered in the first step forward through time. This requires an assumption of no occlusion and no objects being introduced or leaving during an episode. 

I think it's important to discuss and compare against this missing related work [1] --- the question of why non-parametric planning should be conducted over abstract object-centric states versus planning directly on abstract subgoals (which may be less restrictive of an assumption?) is an interesting one.

References
=======
1. Zhang, Lunjun, Ge Yang, and Bradly C. Stadie. "World model as a graph: Learning latent landmarks for planning." International Conference on Machine Learning. PMLR, 2021.
clarity,_quality,_novelty_and_reproducibility: The paper is currently written in a way that makes it difficult to parse what exactly the main claims and technical contributions are. I felt that Section 2 and Section 3 both introduced new ideas beyond what was summarized in the introduction. 

Section 2 purports to provide *a new formulation for goal-conditioned POMDPs*. What exactly about this formulation is *new* with respect to previous literature? References to “standard” goal-conditioned POMDP problem formulation are made but no citations are given. Similarly, mentions of other "standard" POMDP formulations like object-oriented MDPs, relational MDPs, and factorized MDPs are mentioned without citation. My understanding of this section is that it just introduces key “structural assumptions” that are being made about the entities upon which planning is performed?

The paper appears to claim the following as a novel idea: (Section 3) “Our main idea is that generalizing over such a combinatorial space requires the agent to recognize that the same action for moving an object from one location to another can be reused for different objects and in different contexts”. I did not feel as if this claim was properly situated within the context of relevant literature; for example, I would suggest taking a look at [1,2]. Also, in Section 3.1, *"but **our contribution of how we structurally enforce the above three criteria** remains invariant to how the sequential Bayesian filter is implemented, which we expect will advance beyond using SA and GPT"* what exactly is the "contribution" here?

The paper would also benefit from careful editing. There are many sentences that need citations, many typos, and much of the main text is constructed out of long paragraphs with lots of information that are hard to follow. It would also be helpful to clearly define concepts like states, entities, identity, entity-sets, nodes, edges, actions, etc. and provide examples to help explain them early on in the text. 



References
=====
1. Aniket Didolkar, Anirudh Goyal, et al. "Neural production systems." Advances in Neural Information Processing Systems 34 (2021): 25673-25687.
2. Goyal, Anirudh, et al. "Object files and schemata: Factorizing declarative and procedural knowledge in dynamical systems." arXiv preprint arXiv:2006.16225 (2020).

### Other suggestions to improve the presentation
- The four tables presented as a Figure (Figure 6) does not look great and was confusing at first. 

- *“Not modeling this structure means each scene in the combinatorial space of object configurations is treated as **unrelated** from any other scene”* I think “unrelated” is not correct? A scene-level embedding can still learn commonalities between scenes; however, these representations may be entangled with respect to scene objects?

- Some figures need to be resized to fit the page better, as they look too tiny

summary_of_the_review: While the main underlying idea is solid and the general research problem is important and relevant to the community, the technical contribution of the paper appears to be of limited significance. Further improvements to the clarity and quality of the paper may reveal otherwise, but this could require significant revisions.

---
Update: After reading the author's response, they have addressed many of the comments and concerns that were brought up. I have increased my score from 5 to 6.
review_title: null

Meta-review: metareview:_summary,_strengths_and_weaknesses: This paper is about the task of object rearrangement. The paper proposes a neurosymbolic approach for solving this problem. The method is described as a hierarchical abstraction approach-- object representations are learned, and a factorized transition graph is constructed over these representations. Results on the proposed environments demonstrate that the proposed method significantly outperforms the offline RL baselines.

+ The paper presents an interesting new problem;
+ The method proposed has wide potential;

The reviewers suggested to clarify the problem statement, and it has been taken care of. I agree with Reviewer vW5u's suggestions to make the work more transparent, easier for ICLR's audiences to follow. 
justification_for_why_not_higher_score: The limited current scope and the NCS’s design appears to rely too heavily on certain strong assumptions that cater to the synthetic rearrangement environments it is demonstrated on. 
justification_for_why_not_lower_score: The problem formulation is novel, and after revision, the draft is well presented in its current form.
Decision: Accept: poster