Venue: ICLR 2023 Conference

Reviews:

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: The authors present Keypoint Interaction Network (KINet) - an unsupervised learning method to associate keypoints with objects and uses a forward motion model to estimate future keypoint states. As an unsupervised method, the authors make no assumptions about access to ground truth semantic keypoint locations, but rather just use image observations and actions.
strength_and_weaknesses: Strengths:

- Not needing ground truth keypoints is an obvious win. It should make this method usable in multiple setups.
- The keypoint extraction and graph association are object agnostic, meaning that the method should generalize to unseen objects and scenes. That is, the keypoints have no specific semantic encoding that would cause generalization to fail to other classes of unseen objects.

Weaknesses:

- The fixed “K” in the keypoint detector seems particularly problematic. If K is overcomplete (i.e. just some huge value) for describing the intrinsic degrees of freedom of an object, you could argue that perhaps a fixed K is OK, but generally, for articulated or deformable objects, you might not know a priori the “K” value that is needed to fully recover the pose of an object. Another argument to make is that the k-dimensional feature map is just yet another hidden layer of a larger network (that happens to include the graph representations) - in which case it’s probably a stretch claiming these are semantic keypoints at all.
- Related to above, the authors claim that the K representation is a bottleneck, however how much of a bottleneck seems to be very much dependent on the value of “K”.
- Addition of the contrastive loss seems like a tacked-on solution and the authors need to do a better job describing what problem it solves (or what additional regularization it adds). The ablation where it is removed seems fine, but this alone is not enough to justify its use.
- The class of objects the proposed method is evaluated on seems particularly limiting. The real-world results are all roughly rectangular shapes on a consistent background with minimal occlusion (there’s some slightly more complicated shapes in the sim experiments, but even these are simple from an appearance perspective). I would like to see how this method performs on more complex scenes with background clutter and even on articulated or soft-bodied objects. Why is this not unreasonable or out of scope for this paper?: because the promise of this unsupervised method is that it works on any objects without needing semantic labels, however these experimental results fall short of showing that IMO. By contrast, for THIS particular application, you could very likely do simple heuristics to get a set of keypoints that would probably work just as well (e.g. color centroids, etc).
- Nit: The Fig 2 real-robot data does not feel like “robot data” to me (I work primarily in robotics). There’s no robotic embodiment visible in the frame, no active perception, etc. For instance a human could have moved the objects between frames. This is a perfectly fine computer vision dataset, but I think it’s a stretch to claim it’s somehow robot data. 5.2 presumably uses a robot (with MPC on your forward model) to actually move the objects, so why isn’t this visible in the frames?
clarity,_quality,_novelty_and_reproducibility: The paper is well written and easy to follow. Reproducibility seems high (although would be even higher if the source code and simulation data is released - but I didn’t see that mentioned anywhere in the paper).

Experimental results - while limiting (see above) - seem well carried out and of high quality (e.g. multiple seeds with sigma values given).

Originality of the work: There is some novelty in the graph formulation as far as I know. Unsupervised keypoint detection for use in dynamics modeling has certainly been done a few times at this point (e.g. the cited paper Manuelli et al., Keypoints into the future). However I think there’s enough novelty here to justify the submission.
summary_of_the_review: Overall the paper seems novel and is well written. It is likely to be of interest to the community, however I do find the experimental results to be somewhat underwhelming (hence the final score). I’d like to see more real-world examples with more complex object and scene appearances.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: This paper presents a method that learns dynamics models from object-centric representations in manipulation settings. This method is unsupervised and can generalize to different numbers of objects, unseen object shapes and unseen backgrounds. They demonstrate the effectiveness of this forward models by applying it to downstream robotics control tasks with a graph MPC algorithm.
strength_and_weaknesses: Strengths:
1. This paper presents a method to learn unsupervised forward models that can be generalized to different number of objects.
2. The method is straightforward and easy to understand.

Weaknesses:
1. In the related work section, authors mention that Kipf et al.(2019) is very relevant to your method. Kipf(2019) is also unsupervised and doesn't require ground-truth object locations. Could you include this method as part of the baseline as well?
2. Another natural baseline would be Minderer et al.(2019), which also learns a dynamics model base on unsupervised keypoint representations.
3. One of the main contribution is the probabilistic graph representation enabled by the learned adjacency matrix. The insight behind this design is that keypoints that are closer in the space could provide redundant information. However in the visualization of the adjacency matrix(supplement A.2.1), I can't reach to a conclusion as in what's being learned in these graphs. Can you provide more explanation and experiments to prove that this design is essential to the method(other than KINet-deterministic)?
4. I am curious how would the method perform with more keypoints. If there are similar number of keypoints and objects in the scene, each object is only represented by one or two keypoints. Since each object is only represented by their keypoints, is one or two keypoints enough to capture the pose and orientation information?

clarity,_quality,_novelty_and_reproducibility: Clarity:
1. Overall the method and experiments are clear.

Novelty:
1. This method looks very similar to Ye(2020). The main difference in the object representation learning part is that in Ye(2020), they require the ground-truth object locations. The main difference in the forward model part is the probabilistic graph representation. I am not entirely convinced that the novelty is crucial. Presenting more comprehensive baselines and experiments may help. 
summary_of_the_review: This paper presents a new method on unsupervised learning of forward models based on probabilistic graph representations. I am not convinced that the novelties presented in this method are crucial. Presenting more comprehensive comparison to existing methods and other suggested baselines may help.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: This paper proposes an unsupervised framework to learn object-centric representation via keypoint representations. The model abstract the object by keypoints and form a graph representation, where the edges model the relationship between the nodes. An action-conditioned forward model is also learned using contrastive estimation. Experiments are conducted on the rearranging task in MuJoCo as well as a real robot dataset.

strength_and_weaknesses: Strength:
- The idea of combining keypoint abstraction with unsupervised representation learning is novel and the learned forward makes downstream applications like task planning possible.
- The experiments show that the proposed model can achieve high accuracy in forward prediction and the real robot execution task.

Weakness:
- Model design. While combining keypoints and object-centric learning is new, It’s not clear what would be the benefits of choosing a coarser representation as opposed to a denser latent representation. The keypoints abstraction is a bottleneck of the proposed framework, especially in more complex settings or environments. It’s also not obvious how the number of the keypoints $K$ will affect the model's performance as it’s essentially an environment/dataset-dependent hyperparameter. The reconstruction model $f_{rec}$ also depends on $K$. It remains to be seen how the model will perform with more cluttered backgrounds.
- Discussion about more related work. [1] is also a probabilistic object-centric and action-conditioned video prediction model, and interactions between entities are also modeled. [2] Introduces “velocity” and physical interaction and may also be used in modeling action-aware object-centric representations. What are the key differences between the proposed model in terms of the advantages and disadvantages in both representation discovery and forward prediction? 
[1] Veerapaneni R, Co-Reyes J D, Chang M, et al. Entity abstraction in visual model-based reinforcement learning[C].
[2] Kossen J, Stelzner K, Hussing M, et al. Structured object-aware physics prediction for video modeling and planning[J].

clarity,_quality,_novelty_and_reproducibility: Clarity and quality: I find the paper writing and logic flow clear and easy to follow. 

Novelty: The novelty of the proposed model primarily lies in the keypoint abstraction and the interaction network. More justification are expected for why it’s a promising direction.

Reproducibility: Implementation details are not provided in the paper. Code is not included.

summary_of_the_review: My ratings are given based on my evaluation of the strengths and weaknesses above. More justifications are expected in the keypoint-based abstraction and distinction with prior work.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: This paper proposes to use key points as internal representations of scenes and model forward dynamics with graph interaction networks. The authors used a multi-object manipulation testbed for evaluating the proposed Keypoint Interaction Network (KINet). They show that KINet can achieve similar results for forward prediction compared with a baseline graph model with ground truth object positions. The authors also show that KINet converges to the goal configuration faster than baselines and generalizes to unseen circumstances where the number of objects, geometries, and background textures are unseen during training.
strength_and_weaknesses: [+] The paper proposes to extract object key points as representations for modeling forward dynamics. Such an unsupervised learning setting could be treated as a starting step toward many more research areas.

[+] The experiment section is well-organized with not only an evaluation of the future prediction but also a manipulation task.

[-] One major concern lies in the novelty of this paper. The current method shares the same insight as in previous works like OP3[1] and V-CDN [2] since they both adopt an unsupervised key point/mask detection and then a graph message-passing paradigm for solving the dynamics learning problem. Compared with methods, this paper's novelty seems incremental as they can also handle the generalization to an unseen number of objects. The authors should consider adding proper comparison with these two methods as baselines to make a point on the significance of the current model design.

[-] Meanwhile, the currently selected datasets seem simplistic and fail to illustrate the potential of the model on more complex scenes. Under this setting (2D overhead or top view), the results fail to show the effectiveness of model design (e.g. deterministic vs. probabilistic). The authors should consider settings that better justify the significance of their design, especially when other methods showed more complex scenarios like morphing clothes.

[-] A minor question lies in the design for keypoint detection. Why is the current detector taking as input the pair ($I_0$,$I_t$) but not the more common one ($I_{t-1}$, $I_t$)? I don't see a clear advantage in this design since the model will be required to start prediction from a random initial configuration and always comparing with the initial state will cause trouble in long-horizon planning or prediction (i.e., forgetting in long future predictions).

[1] Veerapaneni et al., "Entity Abstraction in Visual Model-based Reinforcement Learning", CoRL 2019.
[2] Li et al., "Causal Discovery in Physical Systems from Videos", NeurIPS 2020.
clarity,_quality,_novelty_and_reproducibility: The paper is generally well-written and easy to follow. However, the novelty of this paper is incremental as supported by the points in the strengths&weakness section. The paper seems to provide sufficient details for reproducing the experiments.
summary_of_the_review: This paper focuses on learning keypoint-based representations for modeling the forward dynamics of a system. However, the proposed method shares similarity with previous works and the design choices seem to have limited significance as shown in a synthetic dataset setting. Therefore, I don't recommend this paper for acceptance for now. I suggest the authors polish the experiment section and design better experimental settings to show the significance of their method compared with the baselines mentioned in strengths&weakness.
review_title: null

Meta-review: metareview:_summary,_strengths_and_weaknesses: This paper jointly trains a keypoint representation and a forward model on it, from observing videos of interacting objects. The keypoint representation loss is based on Kulkarni et al 2019 (Tranporter), and the forward model is a graph-based dynamics model along the lines of interaction networks (Battaglia et al 2016), but with probabilistic edges. 

Strengths:
- During the rebuttal stage, the authors added a new dataset YCBObjs, that contains quite complex shapes, and on which the method performs quite well (Fig 5). Additionally, the improvements during the rebuttal stage included a new and rather recent baseline, that make for a fairly thorough evaluation.

Weaknesses:
- The technical contributions are quite incremental, well approximated as a straightforward combination of Transporter and Interaction Nets.
- The one key difference from this above approximation, the probabilistic edges in the graph, doesn't contribute to measurable performance improvements, as seen in Tab 1 for example (KINet vs KINet - deterministic).
- The idea that this method can work even with novel objects is a bit odd. This is not a claim made by prior work on keypoint detection, such as Transporters, which this builds on. It would be appropriate to be more precise with this statement. What kind of generalization is being claimed here, and is this improving on the generalization that Transporters already provide for keypoint detection? If so, how is this improvement achieved?

Another note to the authors (did not affect my recommendation):
- The right reference for model-predictive control is not Finn and Levine 2017. See e.g. Garcia, Carlos E., David M. Prett, and Manfred Morari. "Model predictive control: Theory and practice—A survey." Automatica 25.3 (1989): 335-348.

- In Sec 5.2, Note that planning horizons of T=40 for these types of tasks are rather long, particularly for pixel-based models such as "Forward" and "Forward-Inverse" among the baselines. 

justification_for_why_not_higher_score: Incremental technical contributions and insufficient evidence to validate them.
Decision: Reject