Learning Preconditions of Hybrid Force-Velocity Controllers for Contact-Rich Manipulation
Authors: Jacky Liang
Xianyi Cheng
Oliver Kroemer
Venue: CoRL 2022 Poster (2022)
Keywords: Contact-Rich Manipulation
Hybrid Force-Velocity Controllers
Precondition Learning

Abstract: Robots need to manipulate objects in constrained environments like shelves and cabinets when assisting humans in everyday settings like homes and offices. These constraints make manipulation difficult by reducing grasp accessibility, so robots need to use non-prehensile strategies that leverage object-environment contacts to perform manipulation tasks. To tackle the challenge of planning and controlling contact-rich behaviors in such settings, this work uses Hybrid Force-Velocity Controllers (HFVCs) as the skill representation and plans skill sequences with learned preconditions. While HFVCs naturally enable robust and compliant contact-rich behaviors, solvers that synthesize them have traditionally relied on precise object models and closed-loop feedback on object pose, which are difficult to obtain in constrained environments due to occlusions. We first relax HFVCs' need for precise models and feedback with our HFVC synthesis framework, then learn a point-cloud-based precondition function to classify where HFVC executions will still be successful despite modeling inaccuracies. Finally, we use the learned precondition in a search-based task planner to complete contact-rich manipulation tasks in a shelf domain. Our method achieves a task success rate of $73.2\%$, outperforming the $51.5\%$ achieved by a baseline without the learned precondition. While the precondition function is trained in simulation, it can also transfer to a real-world setup without further fine-tuning. See supplementary materials and videos at~\url{https://sites.google.com/view/constrained-manipulation/}. 


Reviews:
[{"review_id": "sc-jU-GRWg9R", "reviewer": "Reviewer_6oMV", "summary": "This work proposed a framework to solve contact-rich manipulation tasks, based on learning preconditions of HFVC. The method relaxed the requirements for accurate models and feedback of HFVC. The major contribution is learning the preconditions of HFVC to predict the outcome based on partial point clouds and skill parameters. It can be used for task planning, resulting in higher task performances. Furthermore, the learned model and framework can be transferred from simulated to real environments.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n- It is a promising and meaningful topic to study the combination of nonprehensile and prehensile contact-rich manipulation in constrained environments. The paper proposed a great framework to approach it.\n- The experiments and analysis are solid and thorough. The author compared different ablation studies to demonstrate the effectiveness of each component.\n- The paper is well written and organized.\n- The video greatly demonstrates the performance in action.\n\nWeaknesses:\n- The heuristic is inadmissible for the A* algorithm, but used for practical reasons. It should be mentioned in the main text.\n- Line 96. It is a quite strong assumption that “we assume object geometries are similar to known geometric primitives”.\n- Line 152. “the force control direction, which is chosen to be as close as possible to the robot-object contact normal while being as orthogonal as possible to the desired hand velocity direction.” Does it mean it is an average between the two orientations? What happens if they disagree?\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "The work introduced a framework to solve contact-rich manipulation in constrained environments. The experiments demonstrated the effectiveness of the learning preconditions of HFVS, and its application to search-based task planning. The paper is well written, and the experiments are well conducted.", "technical_quality": "Very Good:  Convincing.", "clarity_of_presentation": "Very Good:  Well organized and clear.", "potential_impact": "", "robotics_focus": "Sufficient demonstration on hardware", "quality_of_limitations": "Limitations are addressed clearly", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"impact": "4: The work contains interesting new ideas that will potentially have major impact in robotics or machine learning.", "issues": "As discussed in Weaknesses.", "reviewer_expertise": "3: The reviewer is fairly confident that the evaluation is correct"}, "extra_text": {}}, {"review_id": "le57poeEQNW", "reviewer": "Reviewer_SLZA", "summary": "This paper presents a way to integrate 3D perception in Hybrid Force Velocity Controllers (HFVC), where force control is used to maintain contact points between the robot and an object and velocity control is used to move the object, for example via a sliding motion. The paper proposes skills of HFVCs that can propose subgoals, and whose parameters are gated by a precondition module that tests when the skill is valid and can be applied. Skills incorporate input from 3D pointclouds, which can be used to get rough object segmentation, and their parameters include the desired object subgoal poses and robot-object contact points. Subgoals are then used in task planning, assuming knowledge of a skill transition model, and the paper evaluates this capability on a real robot with a limited set of simple rectangular or cylindrical objects.      ", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "The main strengths of this paper are (a) showing that compliant control can be achieved via feedback, even with noisy and inaccurate and incomplete object perception, and (b) that one can plan over skills as long as a skill transition model is provided. Additionally, the paper is generally well-written and the robot experiments are convincing.\n\nThe main weaknesses are: \n\nW1: There doesn't seem to be much consideration of force/acceleration limits. In fact, the paper does not provide much detail about the formulation of the optimization problem used to solve HFVCs, given a desired object subgoal pose and robot-object contact points. I would have loved to have seen the mathematical formulation of this problem, currently placed in the Appendix, to be part of the main paper. \n\nW2: Objects are assumed to be rectangular or cylindrical, which is limiting as there are multiple objects that do not fall under these categories (eg plates, cutlery, tools). It would be great to mention what changes would be required if a broader family of objects were allowed.\n\nW3: Only a single object is assumed to be movable in the scene. This is sufficient for showcasing the controller, but does not test for object interaction in cluttered conditions. This is acknowledged in the limitations section, but it would be useful to describe what would need to change to handle multiple moving objects.  \n\nW4: It is not clear what the sim-to-real gap for the training of skill preconditioners in simulation is. If friction and contact are modelled poorly in the simulator, it seems like the preconditioner would also fail. It would be great to have a comparison of preconditioner errors when real vs simulated data is being used. It is not clear if Fig 10 in the Appendix refers to simulated data or real data. I suppose the former.\n     ", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "I am recommending a weak accept based on the fact that the proposed method is shown to be practically useful in non-prehensile manipulation, while using 3D perception. Although the perception system is trained separately from the sequence of HFVC skills, it is still an interesting solution to a challenging robotics problem, that I think the community would be happy to see. I cannot comment with certainty on the novelty of the method with respect to existing literature on HFVCs. If other reviewers suggested rejection, I would consider changing my recommendation.    ", "technical_quality": "Good: Generally convincing, but some additional experiments or analyses needed.", "clarity_of_presentation": "Good: Minor revision needed to improve organization or clarity.", "potential_impact": "", "robotics_focus": "Sufficient demonstration on hardware", "quality_of_limitations": "Limitations are addressed clearly", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"impact": "3: The work is incremental and unlikely to have much impact, even though it may be technically correct and well executed.", "issues": "It would be great if the weaknesses W1-4 are addressed during the revision period. Aside from W4, they do not involve running new experiments, so this should hopefully be doable. ", "reviewer_expertise": "4: The reviewer is confident but not absolutely certain that the evaluation is correct"}, "extra_text": {}}, {"review_id": "Vxky6n-U0pm", "reviewer": "Reviewer_yUsy", "summary": "This manuscript uses hybrid force and velocity controllers(HFVC) as the skill representation and introduces a pipeline to plan skill sequences for contact-rich manipulation. The authors first modify HFVCs to work with inaccurate models and estimate object poses using the robot proprioception for closed-loop feedback. Then the authors propose to learn a point-cloud-based precondition classifier to indicate whether HFVC executions will be successful. Then those learned preconditions are used in a search-based task planner to complete contact-rich manipulation tasks. The authors apply the pipeline to manipulation tasks inside a shelf, achieving a task success rate of 73.2%, significantly outperforming a baseline without the learned precondition.  ", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n1. The paper is well written and easy to follow.\n2. The paper introduces a framework to enable hybrid force-velocity controllers to accomplish various manipulation tasks without knowing the accurate model and object pose feedback. \n\nWeaknesses:\n1. There are many predefined rules in this work. Such as  the “prediction success definition” ( 1) the object moved more than 1.5cm or 20◦ after skill execution, 2) the final pose  is within 7cm and 60◦ of the subgoal pose) It’s not clear whether these rules are generally helpful in a more wide range of manipulation tasks. More questions about these rules are in the Issue section.\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "The manuscript introduces a framework to enable hybrid force-velocity controllers to produce compliant contact-rich behaviors to accomplish various manipulation tasks without knowing the accurate model and object pose feedback. The framework has a wide range of potential applications in the real world. However, there are multiple predefined rules in this work. It’s unclear whether these rules are generally helpful. Therefore, I recommend weak acceptance.", "technical_quality": "Good: Generally convincing, but some additional experiments or analyses needed.", "clarity_of_presentation": "Very Good:  Well organized and clear.", "potential_impact": "", "robotics_focus": "Sufficient demonstration on hardware", "quality_of_limitations": "Limitations are addressed clearly", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"impact": "3: The work is incremental and unlikely to have much impact, even though it may be technically correct and well executed.", "reviewer_expertise": "4: The reviewer is confident but not absolutely certain that the evaluation is correct"}, "extra_text": {"issues": "1. In lines 163-170, it seems that the proposed object pose accuracy also depends on object geometry. It will strengthen the paper if more detailed analyses are provided, such as the errors of different objects. \n2. In lines 192-193, “an HFVC skill execution is considered successful if it satisfies three conditions: 1) the object moved more than 1.5cm or 20◦ after skill execution, 2) the final pose is within 7cm and 60◦ of the subgoal pose” Are the performances sensitive to these parameters?  \n3. The learned preconditions are used for task planners. However, the learned precondition classifier is not 100% accurate. If the task requires several classification results, will the task success rate decrease significantly with respect to the number of precondition classification results? "}}, {"review_id": "F6gaGGJozm9", "reviewer": "Reviewer_4YaQ", "summary": "This paper deals with the problem of manipulating objects under very constraining environmental conditions where non-prehensile skills are imperative to change the objects configuration so that prehensile manipulation is possible. To achieve this, the paper proposes to employ hybrid force-velocity controllers whose activation and sequencing are respectively driven by a learned precondition model and a classical search-based task planner. The proposed framework is tested to manipulate simple-geometry objects through pushing, pivoting, sliding and toppling skills. ", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "**Strengths**\n1. The proposed approach works under more relaxed (realistic) conditions such as partial observations and no closed-loop feedback, making the proposed framework applicable under a broader range of settings. \n\n2. The inclusion of a precondition model, which depends on the point cloud information, which allows the system to assess if the HFVC skill can be executed under the current environment conditions. \n\n3. Performance of the proposed approach when compared to the ablation studies. The proposed framework is extensively evaluated against different ablation conditions, consistently providing better results under different settings. \n\n**Weaknesses**\n1. *Motivation and position of the paper w.r.t state of the art*:\n  * When reading the motivation of this work, something that got my attention is the lack of related works. Aspects as simple as hybrid force-velocity controllers are not accompanied by a reference. More importantly, the full motivation of this work does not refer to any related work where non-prehensile manipulation strategies have been addressed in the past. This gives the impression that they ideas and motivation introduced in this paper are 100% new, which is very far from being true. As an example, one of the first works identifying and addressing the challenges of non-prehensile manipulation is [1], a PhD thesis from 1996. \n\n    [1] K. M. Lynch, \"Nonprehensile Robotic Manipulation:Controllability and Planning\", CMU, PhD thesis. 1996.\n\n  * In lines 59-60, the paper cites 7 papers on non-prehensile manipulation. However, my first concern about this is: why to cite that many papers without providing any insights about them? This, in my opinion, does not bring anything useful to position this paper w.r.t this bunch of 7 papers. Because of this, it is hard to see how useful these references are and how this paper positions w.r.t them. \n\n  * The paper very briefly summarizes learning-based techniques in just a couple of lines (71-72), making special emphasis on their lack of considering multi-step tasks. However, this paper does not bring any contribution on this regard compared to different papers handling long-horizon tasks or skills sequencing, which was not reviewed in this paper. Therefore, I find this review of learning-based techniques a bit unfair, as it is unclear if the review focuses on the learning capabilities of the cited works or the multi-task features. If it is the latter, then the paper is missing to cite a large body of work addressing that specific problem. \n\n2. *Method description*:\n  * Lack of formalism: When introducing the proposed method, the paper disregard mathematical rigor, which is necessary in some parts of the method description. For example, it is not obvious how the objects/robot poses are represented, i.e., do we assume Euler angles, or rotation matrices, or quaternions? This detail is relevant, because depending on the orientation representation, one may need to consider specific constraints when working with different pose representations. In a similar line, there is no formal introduction of the hybrid force-velocity controllers and their interplay with the torque control framework assumed in the proposed method. Therefore, the lack of mathematical rigor leaves too many details open and makes the method description hard to understand in depth. Moreover, when introducing the search-based task planner, the definition of a planning state is unclear. \n\n  * In lines 129-130, the paper describes that robot-object contact points are defined evenly spaced on the surface of the object primitives. However, it is unclear how the geometric shape of the robot end-effector may play a role here (if so). In other words, it seems that the robot-object contact points definition does not consider the geometry of the robot end-effector. If this is the case, this should be said explicitly, if not, then it is unclear how the geometry of the end-effector is considered to define this set of points. \n\n  * In lines 147-155, the paper provides a very high-level (non-formal) description of a quadratic programming problem used to optimize the desired robot velocities. In the main text, there is no mathematical description of how this problem looks like, how it is formulated, and how it is solved. Equally important, this optimization problem seems to be the core of the framework when it comes to compute the desired velocity commands, but no limitations regarding the QP formulation and solution are discussed in the paper. Although the main text refers to the Appendix to know more details about the QP problem, I think an Appendix should serve as complementary contents and in my opinion, the QP formulation is very relevant to understand part of the proposed pipeline. \n\n  * In lines 177-182, the paper reads: \"... we project velocity controls onto the plane that contains the interpolated end-effector trajectory, preventing the end-effector from traveling too far from the intended motion. Note this is not the same as projecting onto the interpolated end-effector trajectory, which would overly constrain HFVC actions and may conflict with force controller commands.\" I read this several times and I still do not understand the motivation for the velocities projection on the plane of the robot's trajectory nor the reason(s) why the latter approach may not work. \n\n  * Several method choices are not justified or motivated. For example, the values used for the precondition success definition are not discussed (do they work for any kind of robot manipulator? any kind of object geometry? etc). Moreover, the paper does not describe how these values were obtained. On a different note, the design of the model architecture is not discussed in detailed. For example, in lines 210-211, the paper describes that the skill parameters are added as additional features to the network, but no motivation is provided here. In summary, several model choices look very arbitrary. \n\n  * Along similar lines to what I have pointed out above, the description of the task planner is very high-level, misses technical details and mathematical rigor. No definition of object/robot poses is given, no definition of planning state is provided, and no formal introduction of the real-time A* algorithm -- the one used in the paper ** is given. \n\n3. *Limitations*:\n  Lines 305-306 mentions some possible limitations coming from the task planner (which is not part of the contributions of this paper, in my opinion). However, the paper does not elaborate under which settings or conditions the task planner is not able to find a solution for tasks where a solution exists. On a different note, the paper does not discuss problems with the perception module, the most important contribution of this paper. Is it really flawless? Last but not least, no discussion about the QP formulation is provided, which is the part of the framework that outputs the desired velocity commands. ", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "This paper provides a very high level description of the proposed method but lacks to provide enough technical details of the problem and designed framework, which makes very hard to assess the contributions and importance of this work. Moreover, the unclear position of this paper w.r.t the state of the art makes it even harder to understand the contributions of this paper. Because of this, the paper requires significant improvements in the method presentation, formalism and positioning w.r.t the state of the art. \n\n-----\n\n**Updated recommendation after rebuttal**\n\nThe revised version of the paper improved in several aspects: the updated related work section does a better job to position the paper w.r.t related works and makes more clear the contributions of this paper; The method description is introduced more formally and the added technical details allow the reader to better understand the technicalities of the proposed framework. \n\nMost of my main concerns were satisfactorily addressed in the rebuttal and revised version of the paper. Therefore I updated my score accordingly.", "technical_quality": "Good: Generally convincing, but some additional experiments or analyses needed.", "clarity_of_presentation": "Good: Minor revision needed to improve organization or clarity.", "potential_impact": "", "robotics_focus": "Sufficient demonstration on hardware", "quality_of_limitations": "Additional details required", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"impact": "3: The work is incremental and unlikely to have much impact, even though it may be technically correct and well executed.", "reviewer_expertise": "4: The reviewer is confident but not absolutely certain that the evaluation is correct"}, "extra_text": {"issues": "When improving this paper, the main points to be address are:\n\n1. Position the paper more clearly w.r.t the state of the art, in terms of the use of HFVC skills, precondition models, and task planners. \n2. Method description is unclear, incomplete and very hard to follow. Mathematical rigor is necessary when introducing variables, optimization problems, task planner and even the design of the neural networks. \n3. A more specific discussion of the limitations of the proposed approach, targeting the specific contributions of this paper. \n\nPlease see my review on strengths and weaknesses above for an elaboration on the foregoing points."}}]

Author rebuttal:

Decision: Accept (poster)