Venue: NeurIPS 2021 Conference

Reviews:

final_score: rating: 4: Ok but not good enough - rejection
review_content: summary: The paper proposes an object-centric transition model that can learn to decompose scenes into object slots and predict action-conditioned slot-wise dynamics end-to-end without supervision. The model has a novel memory-based alignment module that aligns the unordered input object slots with a slot-wise object memory at each time step, so that each memory slot can capture the same object across time and integrate information consistently. This alignment module is trained with an object-level loss rather than a pixel-level loss. The model is evaluated on two datasets featuring partially observable 3D environments. It is shown that the model can outperform OP3 in terms of segmentation quality, can unroll for far more steps than the training regime, and can predict re-appearance of objects. Ablation study shows that the alignment module and the object-level loss are both crucial.
main_review: **STRENGTHS**

- Object permanence plays an important role in human cognition, and is a desirable ability for agents to have so as to effectively interact with 3D environments from first-person views. The paper demonstrates this ability by showing that the learned model can predict re-appearance of objects even after a long period of occlusion.
- The alignment module is well motivated by experiments showing that MONet outputs inconsistently ordered object slots and that with unaligned inputs the transition model performs significantly worse.
- The paper is well written and easy to follow.

**WEAKNESSES**

- The quantitative evaluation seems inadequate:
    - The ARI seems to be computed independently for each time step, which does not measure the slot consistency across time. While there are qualitative results showing the consistency, it would be better to also quantitatively show that the alignment module indeed achieves consistency.
    - It is shown qualitatively that the model can roll out many steps into the future. However, it would be better to also plot a curve showing quantitatively how the performance changes with increasing number of rollout steps.
- The paper failed to discuss the novelty and strength over some relevant work on unsupervised object-centric video models [1-4]. In particular, SCALOR[3] shows ability to maintain consistency, roll out stochastically, and deal with dynamic backgrounds, even when there are around 50 objects. In comparison, the proposed model is only shown to keep track of at most $M=12$ objects. While scalability is not the main focus here, this raises concern about the significance of the paper. Below are two possible ways I can think of to strengthen the paper:
    - Compared to SCALOR, the strength of this paper appears to be that it is action-conditioned and it can predict re-appearance of objects. However, it seems straightforward to extend SCALOR to be action-conditioned, and without experiments, it is hard to tell whether SCALOR will fail to detect objects or predict re-appearance of objects on the datasets in this paper. Hence, I think the paper can be improved by including action-conditioned SCALOR as a baseline.
    - Another potential advantage of the proposed model is that it may be able to recognize the same object when it re-appears at an encoding step. SCALOR cannot do this since the propagation-discovery models will treat objects that re-appear as new objects during inference. Notice that this is more challenging than predicting the re-appearance during rollout, because this requires the memory to indeed not forget the object even after multiple steps of memory updates during which the object is not seen (L128-129 seem to suggest that the memory is fixed during rollout). Therefore, it would be impressive to show results with more (e.g. ~20 in Playroom) encoding steps.

**MINOR COMMENTS**

- It is unclear how many unroll steps are used in Table 1. Is it 6 or 15?
- Footnote 3 is missing (L307).


[1] Exploiting Spatial Invariance for Scalable Unsupervised Object Tracking (AAAI 2020).

[2] Structured Object-Aware Physics Prediction for Video Modeling and Planning (ICLR 2020).

[3] SCALOR: Generative World Models with Scalable Object Representations (ICLR 2020).

[4] Learning 3D Object-Oriented World Models from Unlabeled Videos (OOL Workshop at ICML 2020).
limitations_and_societal_impact: The model may have scalability issues when there are many objects, and it is unclear whether the model can handle object persistence when there are many more encoding steps.
review_title: null

final_score: rating: 5: Marginally below the acceptance threshold
review_content: summary: Current state-of-the-art unsupervised scene decomposition models cannot be directly applied to learn a dynamics model from video. This is because the order of decomposed objects is not consistent across video frames. This paper proposes an alignment module by predicting an affinity matrix that map an orderless output to an ordered output, driven by the consistency loss of the aligned output between timesteps. It can be naturally used to learn the transition model in the level of object, rather than the global scene. They demonstrate the results both quantitatively and qualitatively on synthetic and real world robotics datasets.
main_review: Strength:

The major problem this paper aims to solve is important. The authors propose a surprisingly simple yet effective method to deal with it. I appreciate the simplicity. Meanwhile, in the related work section, the authors make a detailed comparison with previous work, which is helpful to identify the contribution.

Weakness and Suggestions:

1) I have concern about whether the contribution is enough for Neurips. Although the AlignNet is novel and effective, the other two parts of the proposed OAT (Objects-Align-Transition) model are adopted from previous work: the object model is a MONet and the transition model is a simple LSTM. Therefore, I would highly suggest adding experiments on planning / model-based RL tasks for two reasons. Firstly, the main compared work, OP3, did such experiments. It can make a better and clearer argument if the proposed prediction method is better when applied in the RL tasks. Secondly, as the authors argue, one of the most important usage of prediction model is the planning tasks, and showing the planning performance can make a greater impact.

2) The writing needs to be improved. 
From the technical perspective, the authors seems to assume the readers are very familiar with MONet. For example, in section 2.1, it is better to give a concise and self-contained summary of MONet. The current form can easily make readers lost.
From the language perspective: 
- It is mentioned many times the loss is applied at the level of object-structure representation rather than pixels. I can vaguely understand what the authors want to express but I think this sentence itself is confusing: which “pixel-level” method are the authors referring to? If the method is MONet and the loss computation refers to the loss in output space, it seems to be incorrect because the proposed method also use pixel-level loss in the objects model. If the loss computation refers to the loss in the feature space, then which previous work compute feature space loss in the ‘pixel-level’ (or the global level)?
- Some typos: (line 356) out performs; (line 204) because [you] need 

3) In Figure 5, how can the model predict the red object after t=16? It seems the model has no input information about the red object. And What if there is a new object appearing in the scene?
limitations_and_societal_impact: I didn't find any potential negative societal impact of this work.
review_title: null

final_score: rating: 8: Top 50% of accepted NeurIPS papers, clear accept
review_content: summary: This paper introduced a novel way to learn transition models for deep reinforcement learning agents in an unsupervised way, to improve generalization in unseen (out-of-distribution) environments. The main idea is to build a transition model not for individual pixels in the image, which is hard to model or learn, but for a higher-level object representation, because transitions of objects are much more natual and simpler to model and learn for typical 3D environments. To extract object instances from the scene in an unsupervised way, the method ulitizes the existing MONet model (Multi-Object Network) [2] and adds two components:

(1) An alignment module: "Memory AlignNet", which associates frame-by-frame object instances from MONet over time to guarantuee object persistence and identity

(2) A transition module: Transformer + SlotLSTM, which learns to predict transitions of objects, given an action.

With the presented combination of modules, it is possible to roll out future states (given a sequence of actions) more precisely than with existing object-based transition models, specifically in partially observable environments, where knowledge of object persistence and identity is important.

main_review: Pro:
- The paper is generally very well written, the line of argumentation was always clear and the pictures support the reading and message.
- The method outperforms the state-of-the-art baseline (OP3)
- Experiments generally support the arguments well, e.g. why good alignment is important and why a semantically meaningful object-level loss outperforms pixel-level loss
- The results are relevant for the wider RL community

Con:
- Minor: From the very beginning of the paper the term "slot-wise" is used many times, starting in the abstract. To me the term "slot" wasn't clear at all, so maybe it's worth introducing it early in the paper to avoid confusion. I only understood it after reading the original MONet paper.
limitations_and_societal_impact: Yes, limitations have been discussed briefly in the conclusion.
review_title: null

final_score: rating: 6: Marginally above the acceptance threshold
review_content: summary: The paper proposes an architecture for unsupervised object discovery and tracking and shows competitive results on two datasets consisting of video of 3D scenes.
main_review: The paper proposes an architecture for unsupervised object discovery and tracking. The paper proposes a solution to the problem that frame-based object decomposition architectures outputs sets of objects vectors, which are not aligned across time.

The architecture consists of three modules, that are trained jointly, end-to-end. The three modules are 

1) A "standard" MONet image object decomposition module, which decomposes images into sets of object vectors
2) An alignment module "AlignNet", which outputs an alignment matrix A, that aligns new object vectors, z_t with previously seen object vectors.
3) A transition module that predicts new object vectors given past ones.

The main novel part in my understanding is the addition of the AlignNet which serves as a kind of translator between MONet and the transition module. In a sense, it's quite similar to [1], except that [1] doesn't have an explicit alignment module, but rather has a big Transformer model that internally learns to perform the needed alignment between objects between timesteps. I'd encourage the authors to discuss the relation to [1] in their paper.

The paper is fairly well written. There are a few things that could be made more clear. I would appreciate an algorithm describing how the many parts are trained. It's easy to get lost in the text, so summarizing it in pseudo-code would be beneficial. 

It wasn't clear to me how the model handled disappearing objects. In the appendix, it seems the object remembers an occluded object for many timesteps. While this is impressive, it does make me wonder if the model is capable of forgetting objects. There was some text in the appendix on how missing objects are handled, but this wasn't clear to me.

[1] - Ding, David, et al. "Object-based attention for spatio-temporal reasoning: Outperforming neuro-symbolic models with flexible distributed architectures." arXiv preprint arXiv:2012.08508 (2020).
limitations_and_societal_impact: Yes.
review_title: null

Meta-review: This paper presents work on learning an object-centric transition model.  The reviewers appreciated the well-written paper and the proposed alignment module.  However, concerns were raised regarding the quantitative evaluation and novelty with respect to related papers.  The reviewers considered the authors' response in their subsequent discussions.  The reviewers engaged in discussion.  While the problem area is important, the comparisons with respect to other solutions (including SCALOR), the lack of demonstration of object permanence, and overall contribution of the paper remained concerns of the reviewers.

Based on this feedback the paper is not yet ready for publication in NeurIPS.

Decision: Accept (Poster)