Synthesizing Navigation Abstractions for Planning with Portable Manipulation Skills
Authors: Eric Rosen
Steven James
Sergio Orozco
Vedant Gupta
Max Merlin
Stefanie Tellex
George Konidaris
Venue: CoRL 2023 Poster (2023)
Keywords: Learning Abstractions
Mobile Manipulation

Abstract: We address the problem of efficiently learning high-level abstractions for task-level robot planning.  Existing approaches require large amounts of data and fail to generalize learned abstractions to new environments.  To address this, we propose to exploit the independence between spatial and non-spatial state variables in the preconditions of manipulation  and navigation skills, mirroring the manipulation-navigation split in robotics research. Given a collection of portable manipulation abstractions (i.e., object-centric manipulation skills paired with matching symbolic representations), we derive an algorithm to automatically generate  navigation abstractions that support mobile manipulation planning in a novel environment. We apply our approach  to simulated data in AI2Thor and on real robot hardware with a coffee preparation task, efficiently generating plannable representations for mobile manipulators in just a few minutes of robot time, significantly outperforming state-of-the-art baselines.


Reviews:
[{"review_id": "hVcjWoj3gi", "reviewer": "Reviewer_zYUj", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "The paper proposes an algorithm to generate navigation task abstractions that are useful for efficient and precise mobile manipulation.\n\nShows results in a coffee making task on robot.", "summary_of_recommendation": "Weak accept.", "technical_quality": "Good: Generally convincing, but some additional experiments or analyses needed.", "clarity_of_presentation": "Very Good:  Well organized and clear.", "potential_impact": "", "robotics_focus": "Sufficient demonstration on hardware", "quality_of_limitations": "Additional details required", "questions_for_rebuttal": "Add tasks.", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "I find the major contribution of the paper is the idea of having the LLM build navigation/task abstractions that support mobile manipulation.", "extra_scores": {"impact": "3: The work is incremental and unlikely to have much impact, even though it may be technically correct and well executed."}, "extra_text": {}}, {"review_id": "d9Py8Nwpp1", "reviewer": "Reviewer_VDot", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "This paper argues that viewing and representing manipulation and navigation operators/symbols independently and learning navigation symbols/operators. Assuming a fully observable and known world, with full-manipulation-related knowledge (manipulation poses, object properties, etc.) in the world, the robot first makes a manipulation-only plan with the objects and then learns navigation abstractions between manipulation actions, executing a plan at the end that is composed of interleaved manipulation and navigation actions. The paper argues that this approach generates navigation abstractions efficiently compared to an approach where manipulation and navigation are not independent. The approach is verified in a simulation environment and shown to be sample efficient compared to a baseline, and realized in a real robot (where whether navigation abstractions are learned are not is not clear).", "summary_of_recommendation": "The idea of separating manipulation and navigation abstractions is interesting. However, the paper is not well written, the method is not provided clearly and in detail, and the experimental results are difficult to interpret given methodological and baseline-related uncertainties. Therefore, I believe that the paper was not ready for submission in its current form and should be significantly worked on. In its current form, how it does make a significant contribution to the literature is not clear.\n\nUpdate:  I increase my score as the details of the method are more clear, and the idea of separating navigation and manipulation in learning for planning is novel.", "technical_quality": "Good: Generally convincing, but some additional experiments or analyses needed.", "clarity_of_presentation": "Good: Minor revision needed to improve organization or clarity.", "potential_impact": "", "robotics_focus": "Sufficient demonstration on hardware", "quality_of_limitations": "Limitations are addressed clearly", "questions_for_rebuttal": "The approach is compared with a learning-based baseline whose assumptions are not given. Given full observability, which means the exact locations of the objects, their manipulation poses, and the exact layout of the environment are known, does it make sense to compare the method with a standard non-data-driven planning framework?", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "- The Introduction and some related work are well-written however the background is too long and takes up too much (mostly unnecessary) space, in my opinion. \n- The contribution and methodological details are not clearly written. I did carefully read the corresponding parts multiple times and many details are still not precise, mostly due to writing. Let me ask a couple of questions:\n  - l146: How can manipulation-only plans be made without considering navigation? Maybe some objects are in places that are not reachable. \n  - l149: \"v_j represents a region of space in the scene\": what is space?\n  - l150: \"v_j corresponds to the set of poses\": does region correspond to a set of poses?\n  - l152: \"v_j is also referred to as a navigation symbol\": how is a navigation symbol related to a region or set of poses?\n  I am confident that the authors have clear responses to these questions. However, the paper should have been written in a form that is consistent, precise, and clear in the first place.\n- The method details are not given. What exactly is learned? How exactly does learning take place? Which structures are being used in learning? What is the classifier? How is it used? Actually, the paper provides a theoretical framework, provides the motivation, and then does not provide any methodological detail.\n- The experiments and the importance of experimental results are difficult to comprehend. Whether the same assumptions are used by the baseline algorithm is not clear. The exact experiment parameters and the results obtained in the real robot are not provided in the manuscript.", "extra_scores": {"impact": "3: The work is incremental and unlikely to have much impact, even though it may be technically correct and well executed."}, "extra_text": {}}, {"review_id": "QrZeV0nOd9", "reviewer": "Reviewer_Q4vG", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "* The authors propose a semantic map, and related state representation paradigm to allow agent plans to only be specified in terms of manipulation actions---leaving the \"implementation\" detail of navigation actions out of the plan that is specified to the agent\n* In order to do this, the authors assume that manipulation skills, encoded as options in an MDP, are (a) specified w.r.t the reference object frame, and (b) are grounded to feasible locations from which the skills could be performed\n* The authors show that by focusing on the structural parts of the plan that need to be inferred in the moment, in this case, the set of navigation actions required to complete the full plan, and updating state representation to match accordingly, the robot is able to learn the full set of symbols to solve a given planning problem more quickly than if state representation was not appropriately updated.", "summary_of_recommendation": "Conditioned on the suggested edits above, I am happy to accept this paper into the technical program for the conference.", "technical_quality": "Very Good:  Convincing.", "clarity_of_presentation": "Very Good:  Well organized and clear.", "potential_impact": "", "robotics_focus": "Sufficient demonstration on hardware", "quality_of_limitations": "Limitations are addressed clearly", "questions_for_rebuttal": "As the paper currently stands, it is very specific to pick and place (and button-press) type tasks. I recommend that the authors consider a wider class of tasks and posit state decompositions for those classes to make the relevance of this paper more apparent.\n\nI also condition my acceptance of this paper on the inclusion of more details on learning the skill models in the main body of the paper (particularly as the evaluation of the approach is tied to this aspect too).", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "**Strengths**\n\n* The paper hits upon an important facet of robot planning - choosing state appropriately, particularly to leverage the structural properties of the task at hand, can vastly speed up the search problem. In this case, the authors leverage the fact that most _common_ manipulation skills need only be specified by their object frame, thereby allowing plans to generalize across different environments if the navigation aspects are left to be learned.\n* The theory behind the problem formulation is well reasoned and articulated\n* The results are impressive\n\n**Scope for Improvement**\n\n* The authors make a strong, but not necessarily generalizable claim, that an action-oriented sematic map is necessary and sufficient for generating navigation abstractions for mobile manipulation (line 42). The sentiment is expressed elsewhere in the paper. However, this is misleading, as the actions of picking and placing, for which the AOSM is particularly suited, are not the full space of mobile manipulation tasks. Other tasks (task types) exist, such as pushing a cart, inspection (particularly by robots such as Astrobee), collaborative moves, etc. for which a different structural decompositions of state, and hence a different \"map\" might be required to learn the right missing abstractions (not just navigation). I suspect the problem formulation does transfer, but it would be good for the authors to address this fact with appropriate evidence (or walk back their claims).\n* The set of candidate locations for a skill, and the learning of them is crucial to this paper's learning of navigation symbols; yet that is in the appendix. More to the point, Figure 4, is incomprehensible without the content in Appendix 3 (as the terms in the legend are not defined in the main paper). Perhaps the authors should consider moving some of the details of \"abstract representation\" or \"abstract actions\" into the appendix to ensure that the details of skill learning are in the main paper?\n* Lately, there are a new class of planning papers, such as Ahn, Michael, et al. \"Do as i can, not as i say: Grounding language in robotic affordances.\" arXiv preprint arXiv:2204.01691 (2022) or Huang, Wenlong, et al. \"Inner monologue: Embodied reasoning through planning with language models.\" arXiv preprint arXiv:2207.05608 (2022), etc. which provide an alternative to TAMP approaches, particularly for task planning. I recommend that the authors address a comparison to these works as part of their Related Work section as well---particularly how would this work complement (I suspect by providing a grounding(s)?) that work, or surpass it (I suspect in sample efficiency?), etc.", "extra_scores": {"impact": "3: The work is incremental and unlikely to have much impact, even though it may be technically correct and well executed."}, "extra_text": {}}, {"review_id": "47pNITApWx", "reviewer": "Reviewer_yMcH", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "The authors propose to improve the efficiency of learning navigation abstractions that tie into a predefined manipulation plan to complete a mobile manipulation task. The key improvement proposed comes from exploiting structure within a mobile manipulation task that splits the task into manipulation and navigation components. The approach relies on an Action-Orientated Semantic Map with the authors proposing a method for learning the topological graph of nodes and edges with the nodes corresponding to the feasible initiation set of manipulation options. The authors show the success of their approach in simulated experiments and on real hardware for a single coffee preparation task.", "summary_of_recommendation": "The authors proposed method is fairly well presented although lacking a strong motivation. The method appears technically sound with results backing the claims made. Results are presented from simulated and real-world experiments. The method is incremental with the value and contribution appearing to be limited. ", "technical_quality": "Good: Generally convincing, but some additional experiments or analyses needed.", "clarity_of_presentation": "Very Good:  Well organized and clear.", "potential_impact": "", "robotics_focus": "Sufficient demonstration on hardware", "quality_of_limitations": "Additional details required", "questions_for_rebuttal": "-Is the AOSM task specific? For each new task would there need to be a new semantic map?\n-There are some points of ambiguity in the text. For example line 23 the authors state \"the structure\" although it is not clear what structure they are referring to. Same with \"subgoal condition\" line 89. Lines 131-133\n-The order of figures referenced in text is not the same as the order they appear on-page.\n-In line 301, the authors state that the objects are \"scattered around various rooms\", were there multiple rooms? The images and video appear to show a single room.\n-There is a sentence that carries across 6 lines from line 83. It is a bit of a mouthful and hard to follow to the end being so long.", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "Strengths:\n\n- The paper is generally well presented building on previously proposed formulations.\n- The choice of experiments highlight the advantages of the proposed method and answer some logical scientific questions about the method.\n\nWeaknesses:\n\n- The authors do not motivate the need for the proposed method adequately in the abstract and the introduction. \n- The proposed approach appears to be incremental and of limited value.\n- There is a lack of exploration of the limitations that address scaling up the approach to more complicated real-world environments where there are more options, the environment is cluttered and there could be more than 1 task.", "extra_scores": {"impact": "3: The work is incremental and unlikely to have much impact, even though it may be technically correct and well executed."}, "extra_text": {}}]

Author rebuttal:

Meta-review: The paper introduces an approach to leverage manipulation abstractions to learn navigation abstractions. This choice of state abstraction enables an efficient representation for mobile manipulation planning.

Through the rebuttal the authors have engaged with reviewers and improved the paper, and the three reviewers recommend accept, which I concur with. 

The paper could be further improved by (1) creating a full and more detailed algorithm figure and by (2) discussing related work on semantic representations for object search, e.g., https://arxiv.org/abs/2203.10421, https://arxiv.org/pdf/2209.09874.pdf, and https://proceedings.neurips.cc/paper/2020/file/2c75cf2681788adaca63aa95ae028b22-Paper.pdf. Though the paper focuses on a task and motion planning setting, I believe such works are quite relevant to the final state.
Decision: Accept (Poster)