Venue: NeurIPS 2022 Workshop Offline_RL

Reviews:

final_score: rating: 6: Marginally above acceptance threshold
review_content: The idea presented in this paper is straightforward, and conceptually makes sense. However, the paper is full of unexplained terms and concepts (e.g., *symmetric*, *slot attention*, *SLATE architecture*, etc), which renders it hard to follow. It is not good that in the paper the authors explain something by referring readers to other papers. I believe that a good paper should be self-contained. 

In the paper, the authors assume that "*each video transition depicts only a single object being moved*". This assumption is not quite realistic in many real-world scenarios. I am wondering if the proposed approach still applies to the case in which more than one objects are moved simultaneously?

The proposed approach appears a bit complicated. I suggest the authors conduct more ablation studies on different components of the approach to verify each of them.

The environments considered in the experiments seem a bit toy. An easy extension is to replace the simple colored background with natural video background (e.g., Zhang et al., 2021), which would be more convincing.

**References**

Zhang et al., Learning Invariant Representations for Reinforcement Learning without Reconstruction. ICLR 2021.
review_title: An interesting paper with an intuitive idea but hard to read

final_score: rating: 6: Marginally above acceptance threshold
review_content: Summary:

This paper aims to tackle the problem of object rearrangement for embodied agents. The challenges of this problem lie in generalization over different objects and different positions of objects, assuming only image observations. The authors propose to decouple the object representations into object identity representation and object state (i.e., position) representation. They can construct an object-agnostic state transition graph from a fixed set of trajectories. As for experiments, the proposed method is compared with model predictive control, behavior cloning, and implicit Q learning and shows significantly better performance. 

Strengths:

The idea of building an entity-agnostic graph for state transitions is interesting and naturally motivated.

The proposed method is technically challenging. Making the complex pipeline work well is impressive.

The evaluations are conducted on several simulated tasks, and the proposed method is substantially better than the baselines. Also, the appendix includes more analysis of baselines, evaluation settings, and baselines.

Weakness:

One interesting idea of the paper is to learn representations of object identity and object state separately. It will be great to analyze the learned representation qualitatively (or even quantitatively). Is the learned object identity representation fully stable and not affected by the agent's action? Do identity representations for visually similar objects keep close in the representation space? Do the object state representation fully entity-agnostic?

About the state-transition graph, it will be great to show the learned graph in the experiments to help understand it better. 

What are the offline datasets' requirements to make the proposed method work well? For example, if one object location in the test set is not covered in the training offline dataset, could the proposed method generalize for this? Considering that many offline RL papers show experimental results for offline datasets with varying qualities, the paper can be improved a lot by showing the effect of dataset quality.

Though the experiment for "generalizing from an offline dataset of 4 objects to solving rearrangement tasks with 4-7 objects" sounds very exciting, Figure 4 is missing to show this result.

Overall, although there are several weaknesses in evaluation and presentation, this paper is a complete and solid work, so I lean toward acceptance.


review_title: Exploring a challenging problem by a solid approach

Decision: Accept