Venue: CoRL 2023 Conference

Reviews:

final_score: rating: Strong Accept: I recommend accepting the paper and will argue for my recommendation even if other reviewers hold a different opinion.
review_content: summary_of_paper: This work aims at solving long-horizon manipulation tasks by combining the benefits of "learning from demonstration" and "task and motion planning" approaches and proposes "Human-In-The-Loop Task and Motion Planning" (HITL-TAMP), a system that combines automated planning and human control to teach robots complex manipulation skills. 

HITL-TAMP allows a human to manage multiple data collection sessions with robots, improving data collection efficiency. The system also learns planning conditions from a few demonstrations, and the learned conditions are used to guide robots in completing tasks. The system learns planning constraints from a few human demonstrations and uses a queuing system to allow a human operator to manage multiple robots. The collected data is then used to train a TAMP-gated policy using an imitation learning framework.

Experiments showed that HITL-TAMP collected more demonstrations and trained proficient agents faster than a conventional teleoperation system.
review: **Quality**: The quality of the work appears to be high. The authors present a well-motivated and well-executed study that combines task and motion planning with human demonstrations to teach robots complex manipulation skills. The methods are rigorously defined and the experiments are comprehensive, including a user study, various tasks, and real-world testing.

**Clarity**: The paper is well-structured and clear. The authors provide a detailed introduction to the problem, a clear description of the HITL-TAMP system, and thorough explanations of the experiments and results. Diagrams and examples are used effectively to aid understanding. 

**Originality**: The work is original in its approach of combining TAMP with human demonstrations to efficiently collect data for robot training. The proposed system, HITL-TAMP, is a novel contribution to the field of robot learning, providing a solution to the challenge of teaching robots complex, long-horizon tasks.

**Significance**: This work is significant as it addresses a key challenge in robotics: teaching robots to perform complex tasks. The method proposed could potentially be applied in various fields where robotic manipulation is used, such as manufacturing or service robotics.

Strengths:

1. The work provides a novel and effective solution for teaching robots complex manipulation skills.
2. The proposed method combines the strengths of automated planning and human demonstrations, achieving efficient data collection and policy learning.
3. The authors provide a thorough evaluation of the method, including a user study, various tasks, and real-world testing.
4. The method enables a single human operator to manage a fleet of robots, improving scalability.

Weaknesses:
1. The current HITL-TAMP system requires prior knowledge about which task portions will be difficult for TAMP, which could limit its applicability.
2. The system assumes access to coarse object models and approximate pose estimation for real-world applications, which might not always be available.
3. While the authors show that HITL-TAMP can train proficient agents quickly, the long-term adaptability and robustness of these agents in changing or unpredictable environments is not discussed. 

Overall, this paper makes contribution to the field of robotic learning. The authors provide a well-motivated, clear, and comprehensive study that advances our understanding of how to efficiently teach robots to perform complex tasks.
robotics_focus: Sufficient demonstration on hardware
quality_of_the_limitations_section: Limitations are addressed clearly
summary_of_recommendation: This paper introduces a novel system, HITL-TAMP, for teaching robots complex manipulation skills, combining the strengths of automated Task and Motion Planning (TAMP) and human demonstrations. 

Instead of the innovation in the algorithm, this work proposes a well-designed system for collection data, which is a huge contribution to robot learning community.

The authors provide a thorough evaluation of the method, including various tasks and real-world testing, showing that HITL-TAMP improves data collection and policy learning efficiency. 

Therefore, I recommend accepting this paper.
review_title: null

final_score: rating: Weak Accept: I recommend accepting the paper, but will not argue for my recommendation if the majority of other reviewers have a different opinion.
review_content: summary_of_paper: This paper presents a novel data collection and learning system that combines Task and Motion Planning (TAMP) with human teleoperation. It effectively decomposes a task into easier open space movements which can be done by TAMP, and more contact-rich, complex movements that should be done by human teleoperation. Moreover, there is a queuing system that allows human teleoperator to supervises a fleet of robot to enable greater data efficiency. Extensive human user studies are done on the effectiveness of the system.
review: Strengths of the paper: 

1. The problem of enabling efficient data collection is a long-standing and important problem in robotics, and this work that deals with the data collection could be very impactful for future work in fast data collection, which drives robot policy learning on large datasets.

2. The idea of combining TAMP and human teleoperation is novel and useful in two aspects. First, substituting open free space movements with TAMP reduces much human workload of teleoperation and improves efficiency of data collection. Second, it is also a more effective algorithm by combining high-level motor primitives and low-level neural network policy.

3. Extensive study has been done on using the system which involves human operators of different skill levels, which is important for human teleoperation studies.

Weaknesses:

1. The presentation of the method could be clearer - in my understanding, the system composes of two parts: the HITL-TAMP data collection system, and the HITL-TAMP policy. It would be clearer if there could be some more elaborate figure representing the two parts explicitly and the workflow. It is a bit confusing in terms of the presentation as of now.

robotics_focus: Sufficient demonstration on hardware
quality_of_the_limitations_section: Limitations are addressed clearly
summary_of_recommendation: I recommend Weak Accept since both the problem is important and execution of the method and the system is solid. However, it would be great to address some of the questions raised above.
review_title: null

final_score: rating: Weak Accept: I recommend accepting the paper, but will not argue for my recommendation if the majority of other reviewers have a different opinion.
review_content: summary_of_paper: This paper aims to address long-term imitation learning tasks that involve rich contact interactions. The authors propose a framework that incorporates human involvement in the learning process to efficiently gather human demonstrations.

The proposed method divides the execution and learning phases of a given task into two distinct parts. The first part involves minimal physical contact and can be effectively solved using the Task and Motion Planning method. The second part, however, entails contact-rich interactions, necessitating an accurate model and improved sensory precision. As a result, human input becomes crucial in this phase. The authors utilize PDDLStream, a logic-based planning framework, to describe the task's state and the associated contact constraints. Additionally, they employ a hanging task as an example to illustrate how they model the teleoperation actions performed by humans and learn the attachment constraints.


review: Feedback on Pros:

- The paper is technically sound and well-written. The main manuscript is comprehensive, and the FAQ part in the appendix addresses most of the questions that arise after reading the main paper.
- The paper's contribution is based on the recognition that human demonstrations are not needed for all aspects of manipulation. Modern robotics techniques can handle most moving and grasping tasks, while the challenging parts involving contact and friction require human expertise.
- Building upon this observation, the paper presents a framework that efficiently learns the task and collects human skills.
The reported results are reasonable, and the inclusion of videos showcasing both simulation and real robot experiments strengthens the contribution.

Feedback on Cons:

- It would be beneficial to provide a brief description of a conventional teleoperation system in the main manuscript since the main method and discussion rely on this aspect.
- It seems that the conventional teleoperation system represent a group of baseline methods, as shown in FAQ #1 in appendix, then which specific method was used as the baseline in the experiment and why?
- The PDDL Stream section on pages 3 and 4 requires significant effort to read and understand. Although it does not significantly impact my comprehension of the paper, condensing this section could improve the overall readability.

Minor Suggestions:

- Line 111: Replace "p" with "g" to correct the reference to grasp.
- Consider converting Figure 6 into a table format to align with the text.
robotics_focus: Sufficient demonstration on hardware
quality_of_the_limitations_section: Limitations are addressed clearly
summary_of_recommendation: I recommend this paper as it is well written and technically convincing. The main idea and contribution are clear. A few minor issues and confusing parts make me conservative to give a better score. 

Due to my limited knowledge in this field, I would like to see the opinions from the other reviewers to give a more confident recommendation.
review_title: null

final_score: rating: Weak Accept: I recommend accepting the paper, but will not argue for my recommendation if the majority of other reviewers have a different opinion.
review_content: summary_of_paper: The paper proposes to use teleoperated actions within task and motion planning. These non-autonomous actions are specified in the same way as the autonomous actions, namely by specifiying constraints, preconditions and postconditions for PDDL. Following the procedure of Wang et al. [26], some of the constraints can also be learned from teleoperation demonstrations. Using these teleoperation actions, multiple robots can be controlled by a single human, using PDDLStream to plan the next action and putting the robot in a queue, whenever a teleoperation action was chosen. Furthermore, the demonstrations aquired during teleoperation are used to learn a policy using behavioral cloning, such that eventually human teleoperation is no longer required. This paperline is evaluated in a user study showing it can increase the effectiveness of teleoperation and that the learned policies can achieve success rate on long-horizon tasks.
review: (+) It is reasonable to combine TAMP and teleoperation to solve long-horizon tasks that contain difficult segments that require teleoperation. It also makes sense to use these demonstrations to learn policies that avoid the need for teleoperation.

(+) I am also not aware of prior work that used a similar approach to combine TAMP and teleoperation.

(-) However, the contribution seems to be quite small. It is not clear to me what would be the main takeaway: That we can use teleoperation within task and motion planning? That we can control multiple robots in parallel by putting them into a queue? That we can use the demonstrations for imitation learning? All of this seems to be quite obvious. The specific pipeline could be an important contribution, if it was less limited. The overall idea is implemented rather naively, for example, by using simple behavioral cloning for learning the policies. As a consequence, it seems hard to believe that the pipeline could be scaled to real applications without substantial changes. 

(-) Another issue with the current submission is that the approach is not sufficiently well explained. While the writing is good and easy to comprehend, many important details are not at all described. For example, for learning the constraint the paper only states that the constraints are modeled as distributions and how the data set is collected, but the paper does not state how the distributions are modeled and optimized. While I assume, that the proposed method closely follows the prior work [26], that would need to be stated more explicitly, and also the approach should be stated in a bit more detailed so that the reader is not completely lost without consulting the prior work. Also, important definitions are missing. What is the grasp g? Is it an SE-3 pose? The paper states that "forward kinematics f rom configuration q multiplied with grasp p produces pose p". Does this refer to a multiplication of homogenous transformation matrices? That would mean that the grasp is specified relative to a given end-effector pose? Furthermore, the experimental setup needs to be explained in much more detailed. How was the user study conducted (What instructions where given, which setting was tested first, what exactly where the state-action spaces in each experiment?)
robotics_focus: Sufficient demonstration on hardware
quality_of_the_limitations_section: Additional details required
summary_of_recommendation: The overall idea is reasonable, but the submission lacks a clear contribution. Furthermore, the whole pipeline needs to be presented in more detail.
review_title: null

Meta-review: The authors' responses to the reviewer's comments have been incorporated into the meta review. The paper introduces a framework that integrates teleoperated actions with task and motion planning (TAMP) for addressing long-horizon tasks containing challenging segments that require human intervention. The proposed approach combines TAMP and teleoperation to effectively switch between autonomous and human-guided actions, with the goal of learning policies that reduce the reliance on teleoperation over time.
Strengths:
1.	The reviewers recognize the rationality of the approach, merging TAMP and teleoperation to solve complex tasks that involve both autonomous and human-guided actions.
2.	The integration of learned constraints, TAMP-based actions, and teleoperated actions to improve the efficiency and autonomy of the system is appreciated.
3.	The evaluation of the approach through a user study and the demonstration of improved teleoperation effectiveness, along with the policy learning for autonomous execution, are highlighted as valuable contributions.
Weaknesses:
1.	The contribution of the paper is perceived as small, and the main takeaways are not clearly highlighted.
2.	The implementation of the proposed pipeline is considered to be relatively basic, and its scalability to real applications is questioned.
3.	The presentation of the approach and important details is deemed insufficient.
The reviewers acknowledge the novel combination of TAMP and teleoperation, but they express concerns about the clarity of the main contributions and the limited scope of the implementation. They note that while the idea of the approach is reasonable, the paper needs to better define its contribution and explain the specific methodology. Additionally, the reviewers request further elaboration on the constraint learning process, the experimental setup, and the definitions of key terms.
In response, the authors address these concerns by emphasizing the contributions in the introduction and clarifying the unique aspects of their approach. They explain the methodology more explicitly, particularly regarding constraint learning, and provide additional details about the experimental setup. Reviewers now revised recommendations to accept.
Decision: Accept (Poster)