Learning Generalizable Manipulation Policies with Object-Centric 3D Representations
Authors: Yifeng Zhu
Zhenyu Jiang
Peter Stone
Yuke Zhu
Venue: CoRL 2023 Poster (2023)
Keywords: robot manipulation
imitation learning
object-centric representations

Abstract: We introduce GROOT, an imitation learning method for learning robust policies with object-centric and 3D priors. GROOT builds policies that generalize beyond their initial training conditions for vision-based manipulation. It constructs object-centric 3D representations that are robust toward background changes and camera views and reason over these representations using a transformer-based policy. Furthermore, we introduce a segmentation correspondence model that allows policies to generalize to new objects at test time. Through comprehensive experiments, we validate the robustness of GROOT policies against perceptual variations in simulated and real-world environments. GROOT’s performance excels in generalization over background changes, camera viewpoint shifts, and the presence of new object instances, whereas both state-of-the-art end-to-end learning methods and object proposal-based approaches fall short. We also extensively evaluate GROOT policies on real robots, where we demonstrate the efficacy under very wild changes in setup. More videos and model details can be found in the appendix and the project website https://ut-austin-rpl.github.io/GROOT.


Reviews:
[{"review_id": "yiEvhT5WkSG", "reviewer": "Reviewer_ghms", "summary": "", "questions": "", "limitations": "", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "This paper proposes GROOT, an imitation learning approach for visual-based manipulation by learning structured 3D object-centric representations. They utilize a transformer-based policy over point-clouds, structured to utilize the object-centric inductive bias, leading to performance gains in manipulation tasks, both in simulated and real environments. At inference time, to allow generalization, they use a general segmentation module (SAM) to segment the entire observation, and utilize a pre-trained representation module (DINO) to find the objects-of-interest via cosine-similarity and propagate them to the trained network.", "summary_of_recommendation": "Overall, I like the idea of this paper and I’m convinced that this approach can help generalization. In addition, simulation and real robot experiments show that indeed the object-centric approach is crucial. Given my questions are addressed, I would like to recommend this paper for acceptance.", "technical_quality": "Excellent: Very convincing and thorough.", "clarity_of_presentation": "Very Good:  Well organized and clear.", "potential_impact": "", "robotics_focus": "Sufficient demonstration on hardware", "quality_of_limitations": "Limitations are addressed clearly", "questions_for_rebuttal": "* Backprojection to PCs: is it done via an external library (Open3D) or something else? I see you cite it but it is not clear what is the input and output of that step.\n* Transformer input (lines 189-198): if I understand correctly, each object is represented as a PC, which is then clustered to groups. Each group is fed through PointNet++ with global pooling to output a single vector which is then finally used as input to the transformer. It would be beneficial to clarify this step with mathematical notations, i.e., the inputs and outputs dimensions. Is the number of clusters (e.g., 3) per PC is predetermined (a hyper-parameter)?\n* Positional encoding in the transformer: at timestep $t$, tokens (=clusters) of the same object get the same temporal encoding? Similarly, different objects but both at timestep $t$ get the same temporal encoding? Several object-centric works considered different positional embeddings to better capture spatio-temporal relationships between objects ([1]-[3]).\n* Comparison with VIOLA: in essence, GROOT replaces the RPN for object proposals with user-interaction and S2M to select objects and utilizes depth to backproject to point-clouds. GROOT also inherits the idea of masking tokens. VIOLA also mentions that they use multiple views to handle occlusions. While the authors mentioned that for a fair comparison they modified VIOLA to support depth input (RGB-D), have you tried using VIOLA with multi-view input as a baseline? Alternatively, instead of user-interaction, have you considered using RPN similarly to VIOLA? I believe that a hybrid model of VIOLA and GROOT can lead to another improvement (though I don’t expect the authors to implement it).\n* To facilitate further research, are you planning to make the code available?\n\n[1] https://arxiv.org/abs/2107.09240\n\n[2] https://arxiv.org/abs/2210.05861\n\n[3] https://arxiv.org/abs/2306.05957\n", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "Strengths:\n* \\+ real robot experiments.\n* \\+ strong performance, significant gains over baselines.\n* \\+ generalization capabilities; though it requires another pre-trained representation model at inference time (DINO).\n* \\+ converting RGB-D to PC is an interesting approach to allow generalization in manipulation tasks, avoiding distracting signals in images such as colors and brightness/contrast.\n\nWeaknesses/Limitations:\n* \\- while the user interaction is rather minimal, I find the requirement to annotate/segment by the user for each scene a limitation of this method.\n* \\- The approach proposed in this paper employs several pre-trained modules such as S2M, SAM, DINO and XMem. While I don’t find this a significant limitation, I suggest that for better clarity the authors will organize the pre-trained and training models in a table (or something else) to help the reader understand which parts of the model are trained and which are frozen. For example, a table with columns: Model (e.g., S2M, DINO), Purpose (e.g., Segmentation, Tracking), Pre-trained/Trained, etc…\n\nMinor Comments:\n* Lines 258-259: “; 2) Three variants for generalization tests: , Camera-Shift, and New-Object. “, I believe the “Background-Change” variant is missing.\n* Line 280: missing word after “and”: “The comparison between VIOLA and [missing] suggests”.\n", "extra_scores": {"impact": "4: The work contains interesting new ideas that will potentially have major impact in robotics or machine learning."}, "extra_text": {}}, {"review_id": "Tmq7K9oxDkT", "reviewer": "Reviewer_p4GH", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "The paper proposes a method called GROOT for learning generalizable manipulation policies using object-centric and 3D representations. The main contributions of the paper include the construction of object-centric 3D representations robust to background changes and camera views, the use of a transformer-based policy for reasoning over the representations, and the introduction of a segmentation correspondence model for reusing strategies in the case of new object manipulation. The paper presents comprehensive experiments in simulated and real-world environments to validate the effectiveness and robustness of the proposed method.", "summary_of_recommendation": "Very solid experiments section but some ideas are not expressed clearly due to the lack of explanation.", "technical_quality": "Very Good:  Convincing.", "clarity_of_presentation": "Excellent: Enjoyable to read.", "potential_impact": "", "robotics_focus": "Highly relevant to robotics but no hardware experiments", "quality_of_limitations": "Limitations are addressed clearly", "questions_for_rebuttal": "1. Have the authors experimented wit other point cloud encoders?\n2. Is it possible to reduce human input during training data generation?\n3. Would it be possible to explain intuitively why GROOT performs better at generalization?", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The paper introduces a novel approach, GROOT, which combines object-centric and 3D priors to address the challenges of generalization in perception for manipulation tasks. This approach takes advantage of structured visual representations to enhance the robustness and adaptability of visuomotor policies. GROOT uses state-of-the-art visual recognition models and interactive segmentation techniques to construct object-centric 3D representations. It leverages depth information and back-projects segmentation masks into 3D point clouds, capturing object information and spatial relations in 3D space. The paper provides a comprehensive evaluation of GROOT in both simulated and real-world environments. The results show that GROOT outperforms baseline methods in terms of generalization abilities, handling background changes, different camera angles, and new objects. The training process, however, is rather manual, which requires manual labor of segmenting the images to annotate the training data. Furthermore, the paper lacks an intuitive explanation of why the method performs better than baselines such as VIOLA, as most of the components in the system are \"standard\".\n", "extra_scores": {"impact": "3: The work is incremental and unlikely to have much impact, even though it may be technically correct and well executed."}, "extra_text": {}}, {"review_id": "8OgGMPJEtu_", "reviewer": "Reviewer_YKj9", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "This paper proposes an imitation learning method aimed at learning robust policies that generalize beyond their training configurations. They construct an object-centric 3D representation by segmenting out objects in a point cloud and using them to condition an imitation learning policy. The authors argue that incorporating structural visual representations into the design of an imitation learning method is key to handling visual variations in the real world, and propose a method for transforming input point clouds to a canonical world frame to tackle varying camera viewpoint changes. The authors show experiments on tasks in simulation as well as the real world, as well as perform generalization experiments to show invariance against background changes, camera viewpoints, and new object instances.", "summary_of_recommendation": "The paper needs a major revision as there are multiple missing words, typos, grammatical errors, as well as some technically incorrect statements. There are also some missing discussions about limitations of the proposed method in my opinion, which also bring the impact of this work on real-world robot deployment into question. Additionally, the experimental results (doing worse on canonical than when perturbed) make it seem that there is potentially a performance gap between the current state of the method and what it’s ideal implementation would be.", "technical_quality": "Fair: Significant additional evidence required to support the claims.", "clarity_of_presentation": "Fair: Major revision needed to improve organization or clarity.", "potential_impact": "", "robotics_focus": "Sufficient demonstration on hardware", "quality_of_limitations": "Additional details required", "questions_for_rebuttal": "1. Table 1 seems to show that for all tasks, GROOT had a better success rate with some environmental perturbations than in the canonical setup. Same case for success rates on two real-robot tasks, where the proposed method performed worse in the canonical setup than when background was changed. How was that possible? \n2. One clear limitation of the work to me is the need for camera extrinsics for transforming input point clouds to canonical world frame. The need for camera-viewpoint generalization in learned policies is usually the lack of such transformations being readily available, and the expectation from the model to learn them implicitly. Can the authors discuss this limitation in detail and its implications when deploying their method on a real robot?", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "Quality: This paper addresses an important generalization challenge in robot learning, providing a reasonable algorithm that bakes in object and camera-viewpoint into the method. While the methodology is reasonable, some experimental results and evaluation setups can be under question.\n\nClarity: The paper is well-structured, however there are many typos and missing words, in addition to some technically incorrect statements.\n\nStrengths:\n1. The paper targets a major challenge in robotics which is generalization to visual scene perturbations and camera viewpoint changes. The multi-step methodology for segmenting out objects of interest and conditioning the policy using point clouds always transformed to the same world frame is an interesting method for robotic manipulation which can generalize to camera viewpoint changes. \n\nWeaknesses:\n1. Line 126: The authors mention that their “objective is to find a visuomotor policy $\\pi$ that maximizes the expected task success rates.” This is incorrect, as for the behavior cloning policy proposed in this paper the objective is to mimic the expert and there are no rewards obtained from the environment that the agent can maximize.\n2. Line 257: missing keyword for first generalization test\n3. The method needs access to ground truth camera extrinsics in order to transform the observed point clouds to a canonical world frame. This might not always be easy to obtain in real world setups.\n4. In the video demos, the initial conditions seem to be very insignificantly perturbed. For camera-viewpoint generalization, while the model is shown to work with observations from varying camera positions, all evaluation rollouts act on almost the same initial setup which could have been cherry-picked.", "extra_scores": {"impact": "2: Unsure, difficult to assess."}, "extra_text": {}}, {"review_id": "-tNdgdgeiz", "reviewer": "Reviewer_Yp2B", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "\nThe paper introduces GROOT, an approach to imitation learning that uses a transformer backbone in combination with object embeddings obtained from standard large vision models. Roughly, there are two stages: vision preprocessing and then IL using the transformer. \n\nIn the preprocessing step, the human user provides demonstrations wherein the task-relevant objects are identified using a scribbling interface. These objects are tracked through the rest of the frames of the demonstration using a tracking model. At time time, the model attempts to find the same objects as from the demonstrations using dino2 features.\n\nIn the IL step, the model uses a transformer where the tokens are 3d models of the objects obtained by projecting the test-time object segments onto the point cloud. The point cloud object representations are clustered and tokenized. A standard policy transformer is used where a specific action token is introduced to generate the action.\n", "summary_of_recommendation": "I didn't see many novel ideas here. Nevertheless, the method outperforms VIOLA on three relevant tasks, so there is a contribution.", "technical_quality": "Very Good:  Convincing.", "clarity_of_presentation": "Good: Minor revision needed to improve organization or clarity.", "potential_impact": "", "robotics_focus": "Sufficient demonstration on hardware", "quality_of_limitations": "Limitations are addressed clearly", "questions_for_rebuttal": "I didn't see anywhere in the paper where you mention the number of demonstrations used in your experiments. That's information you should probably include.", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "While I think that the performance of this model is pretty good relative to VIOLA, I think this fact can be attributed to the better vision models used here. I do not see very many novel ideas in this paper. The policy backbone (a big transformer with a standard configuration) seems similar to many other works. The preprocessing on the input seems to be a basic application of the vision models, i.e. object detection, segmentation, dino2, etc. \n\nIn terms of results, the model performs pretty well, outperforming VIOLA in situations where there is a significant shift in background or camera perspective.\n\nIn summary, I think this is notable work, but the lack of novelty dampens my enthusiasm. It would have been good to expand table 1 to include results from a larger number of tasks.", "extra_scores": {"impact": "3: The work is incremental and unlikely to have much impact, even though it may be technically correct and well executed."}, "extra_text": {}}]

Author rebuttal:

Meta-review: The authors present an imitation learning method for visual-based manipulation. They use object-centric 3D representations (segmenting out objects in a point cloud) that encode invariance to background appearance and camera position in order to improve robustness of a downstream imitation learning policy. They call the algorithm GROOT and they demonstrate compelling results on both simulated and real-world robotic environments, including ablations to measure generalization and invariance to various distractor features (i.e. background), latent variable variations (e.g. camera pose) and to new object instances.

Reviewers highlighted the importance of generalization for imitation learning as a problem, likely as a result of the author’s clear and concise motivation of the problem itself. (aside: not raised by reviewers, but this AC does wonder if such generalization issues are a result of the small scale of the paper’s datasets, which are mitigated by the recent trend to large internet-scale cotraining). Several clarification issues were raised during the rebuttal process and the authors provided detailed explanations for missing information and additional experiments where appropriate. One reviewer raised their score as a result. The resultant recommendation from all reviewers was towards acceptance.

One reviewer highlighted that the choice to have users annotate and segment each scene is a limitation of the method; a concern that was reiterated at the end of the rebuttal process (after author rebuttals). With that said, this seems like something that could be automated for future work and seems very addressable with some effort - as highlighted in the author’s response (aside: the author’s argument that this user-interaction step increases transparency and user trust would need to be verified).
Decision: Accept (Poster)