Object-Centric Dexterous Manipulation from Human Motion Data

Authors: yuanpei  chen
chen  wang
yaodong  yang
c karen liu
Venue: robot-learning_org_CoRL_2024_Conference
Published: 2024-06-06 08:51:22

Abstract: Manipulating objects to achieve desired goal states is a basic but important skill for dexterous manipulation. Human hand motions demonstrate proficient manipulation capability, providing valuable data for training robots with multi-finger hands. Despite this potential, substantial challenges arise due to the embodiment gap between human and robot hands. In this work, we introduce a hierarchical policy learning framework that uses human hand motion data for training object-centric dexterous robot manipulation. At the core of our method is a high-level trajectory generative model, learned with a large-scale human hand motion capture dataset, to synthesize human-like wrist motions conditioned on the desired object goal states. Guided by the generated wrist motions, deep reinforcement learning is further used to train a low-level finger controller that is grounded in the robot's embodiment to physically interact with the object to achieve the goal. Through extensive evaluation across 10 household objects, our approach not only demonstrates superior performance but also showcases generalization capability to novel object geometries and goal states. Furthermore, we transfer the learned policies from simulation to a real-world bimanual dexterous robot system, further demonstrating its applicability in real-world scenarios. Project website: https://cypypccpy.github.io/obj-dex.github.io/.

Main paper: 
 introduction: 
 Developing bimanual multi-fingered robotic systems capable of handling complex manipulation tasks with human-level dexterity has been a longstanding goal in robotics research. When a robot engages general manipulation tasks beyond a simple pick-and-place, the definition of a task can be difficult to establish. Previous works defined manipulation tasks in a variety of ways, including the state of the environment [1, 2] , symbolic representations [3, 4] , or language descriptions [5, 6] . Regardless of how the goals are specified, a common element across these definitions is an object-centric perspective focusing on the state of the objects being manipulated. As such, the goal of our work is to train a policy for a bimanual dexterous robot to manipulate the objects according to the task goal defined as a sequence of object pose trajectories.Prior works primarily utilize deep reinforcement learning (RL) to learn object-centric dexterous manipulation skills [7] [8] [9] . Despite the success of these methods in tasks such as in-hand object reorientation [10] [11] [12] , they typically focus solely on learning the movements of the fingers, neglecting the integrated coordination of both arms and hands. Training RL policy that controls both robot arms and two multi-finger hands is possible in theory, but presents substantial challenges in practice due to the high degree of freedom of the robot action space. Imitation learning (IL) can potentially tackle this challenge by leveraging the guidance from human motion data to assist policy learning. However, another challenge arises due to the morphological differences between human and robotic hands, often referred to as the "embodiment gap". For instance, some robotic hands are designed with only four fingers, each significantly larger than a human's, making it difficult to retarget the human hand trajectories to the robot hand while achieving the intended manipulation tasks.One critical observation is that human finger motions are not consistently useful across various manipulation tasks due to the embodiment gap. In contrast, human wrist motions offer valuable information less sensitive to the embodiment gap, such as where to place the palm and how to interact with the objects in 3D space. Such motion cues significantly reduce the complexity of the high-dimensional action space in RL training, allowing it to focus on exploring finger motions to achieve the object-centric task goal. Based on this observation, we propose a hierarchical policy learning framework consisting of a high-level planner for the wrist and a low-level controller for the hand. The high-level planner is a generative-based policy, trained by imitation learning with human wrist movements, to generate robot arm actions conditioned on a desired trajectory of the object's movements. Based on the generated arm motions, the low-level controller outputs fine-grained finger actions learned through RL exploration rather than imitation of human data. The reward function for the RL training is the likelihood between the object's movements during interaction and the reference trajectory. By harnessing the strengths of both RL and IL, we enable the robot to adapt to its own hand embodiment while keeping the training tractable by refining the action space using human data.To ensure that the learned policy can adapt to a variety of reference object trajectories, not just a single scenario, we utilize ARCTIC [13] , a comprehensive dataset with 51 hours of diverse hand-object manipulation motion capture sequences. Our experiments demonstrate that the learned policy exhibits generalization to novel object geometries and unseen motion trajectories. In addition, we successfully transfer our policy from simulation environments to a real-world bimanual dexterous robot, further validating its practical applicability in real-world manipulation tasks. 
dexterous manipulation: 
 Dexterous manipulation is a long-standing research topic in robotics [14] [15] [16] [17] . Traditional methods rely on analytical dynamic models for trajectory optimization [14, 15, 17, 18] , which fall short in complex tasks due to the simplification of contact dynamics. [19] and [20] propose model-based optimization methods to solve the contact-rich dexterous manipulation task, but a correct dynamics model of the environment is required. Recently, deep reinforcement learning (RL) has showcased promising results in training dexterous manipulation skills such as in-hand object reorientation [10] [11] [12] [21] [22] [23] [24] [25] [26] [27] [28] , bimanual manipulation [7, 8, 29] , sequential manipulation [30] [31] [32] , and human-like activities [33] . [24] learn a grasping policy to make the object follows a manual designed trajectory, while we focus on more challenging tasks. Despite the progress, successfully training a dexterous RL policy often requires extensive reward engineering and system design, which limits its practicality in some scenarios. Besides RL, imitation learning (IL) is also widely used for training dexterous policies [34, 35] . By performing supervised-learning with human teleoperation data [36] [37] [38] [39] [40] , prior works show impressive results in dexterous grasping [41, 42] and general manipulation tasks [43] [44] [45] [46] [47] [48] [49] [50] . However, teleoperation data are often expensive to collect due to the requirement of a real-world robot system.Recently, learning from human motion data has started to receive more attention because it allows scaling up data collection without robot hardware. Prior works leverage human videos [51] [52] [53] [54] [55] [56] , motion capture data [57] [58] [59] [60] [61] [62] to extract valuable motion hints for manipulation such as trajectory-level plans [51, 59] , object affordance [53] and motion priors [52, 55] . For dexterous manipulation, VideoDex [55] , DexMV [43] , DexTransfer [58] and DexCap [63] showcase the potential of using analytical methods (e.g., inverse kinematics) to retarget human hand motion to robot hardware, such as matching joint angles [43, 55] , fingertip positions [63] . [64] showing independent control of the fingers and arm for capturing detailed hand manipulation of objects. DexPilot [38] formulate the retargeting objective as a non-linear optimization problem and retarget human motion to robot by minimizing a cost function. However, due to the embodiment gap between human and robot hands, position-based retargeting methods do not guarantee the replication of task success. In contrast, our approach uses human data as guidance for RL training, which learns the motion retargeting conditioned on the robot's embodiment. Notably, [34, [65] [66] [67] [68] [69] share the same idea of utilizing human data as guidance or reward for reinforcement learning. While these works focus on learning human whole-body control in simulation, we study dexterous manipulation with multi-fingered robotic hands and transfer the learned policy from simulation to the real world. 
task formulation: 
 The goal of an object-centric manipulation task is to let the robot physically interact with the object to achieve the desired motion trajectory. We define the motion trajectory as the sequence of the object's SE(3) transformation G = (g 1 ,g 2 ,...,g T ), where each time step g i = (g R i ,g T i ,g J i ) consists a 3D rotation g R i , a 3D translation g T i , and the joint angle g J i . g J i can be omitted if the object is a single rigid body. We then formulate an object-centric manipulation task as a Markov Decision Process (MDP) M = (S S S,A A A,π,T ,R,γ,ρ,G), where S S S is the state space, A A A is the action space, π is the agent's policy, T (s t+1 |s t ,a t ) is the transition distribution, R is the reward function, γ is the discount factor, and ρ is the initial state distribution. The policy π conditions on the reference object state trajectory G and the current state s t , and generates robot action distributions a t to maximize the likelihood between the future object states (s t+1 ,s t+2 ,...,s t+T ) and the reference trajectory G. This formulation is versatile to adapt to downstream tasks, such as moving objects to desired locations and in-hand object re-orientation with bi-manual hands. 
method: 
 In this section, we introduce our framework for object-centric manipulation. The overview of the framework is shown in Figure 2 . Our framework consists of three parts: high-level planner (Section 4.1), low-level controller (Section 4.2) and the data augmentation loop (Section 4.3). The details of our sim-to-real policy transfer are introduced in Section 4.4.One of the key challenges in training a dexterous robot policy, especially with bi-manual dexterous hands, is managing the high-dimensional action space. Despite the embodiment gap between human and robot hands, human hand motion, particularly wrist movements, provides highly informative hints for how to interact with objects and environments. Such motion hints not only narrow the action space for the robot but also provide guidance toward the task goal. Based on this observation, we first train a generative model to synthesize wrist motion by imitating human wrist movements in the motion capture data.We train a Transformer-based generative model π H that takes object category ID c, and the desired object motion trajectory G = (g t ,g t+1 ,...,g t+T ) as inputs and outputs a sequence of 6-DoF wrist actions (a W t ,a W t+1 ,...,a W t+T ), where each action a W i = (p l i ,p r i ) consists the 6-DoF pose of the left hand p l i and right hand p r i in SE(3). In our experiments, we use T = 10. We leverage the entire ARCTIC dataset [13] to train π H with a behavior cloning algorithm. The total training samples consist of 339 sequences of human hand mocap trajectories, totaling 2.1 million steps.Based on the generated wrist trajectory from the high-level planner π H , the low-level controller π L can start from a reasonable wrist pose and focus on learning the fine-grained finger motions to physically interact with the object to achieve the task goal G. We use Proximal Policy Optimization (PPO) [70] to train π L . The policy π L takes the current observation s i , the desired object motion trajectory G = (g t , g t+1 , ..., g t+T ), and a sequence of 6-DoF wrist actions (a W t , a W t+1 , ..., a W t+T ) generated by high-level planner as inputs, and outputs the finger joint action a F t . Here the observation s t contains the object pose and robot proprioception. The reward function is defined asr t = exp -(λ1 * ∥g R t -ĝ R t ∥2+λ2 * ∥g T t -ĝ T t ∥2+λ3 * ∥g J t -ĝ J t ∥2) , aiming to minimize the distance between object's movements and the desired goal trajectory. ĝR t , ĝT t and ĝJ t is the current 3D translation, 3D rotation and joint angle of the object respectively. λ 1 , λ 2 , and λ 3 are the hyperparameters to balance the weight of each component of the reward. In some scenarios, when the robot's hand is larger than the human's, the generated wrist actions from the high-level planner π H need to be adjusted accordingly. To achieve this, π L learns to output a residual wrist action ∆a W t within a fixed range (±4 centimeters for transition and ±0.5 radian for rotation). With the residual wrist actions, the policy can now adjust the wrist position to better synthesize the motion of the robot hand. The final robot action is a combination of (a W t +∆a W t ,a F t ). Please refer to Appendix B for more detail about the observation space and the reward function.Training policy only with the ARCTIC dataset [13] limits the diversity of task objects and generated motions. For instance, given a box twice the size of the one used in the dataset, it is challenging for the When deploying the policy to the real world, some observation cannot be accurately estimated, such as joint velocity and object velocity. We use the teacher-student policy distillation framework [11, 21, 71] to remove the dependence on these observation inputs from the policy. In real-world deployments, our system uses four top-down cameras to perform articulated object pose estimation. We use FoundationPose [72] to estimate the 6-DoF pose of different parts of the articulated object and calculate the joint angle between them. The entire object pose tracking system runs at the speed of 15Hz. The learned policies generate actions to control the robot, and the low-level controller is processed by a low-pass filter with an exponential moving average (EMA) smoothing factor [11] to reduce the jittering motions of the robot fingers. For more details on the sim-to-real setups, please refer to Appendix C. 
experiments: 
 The experiments are designed to answer the following research questions: (1) Can the high-level planner generalize to unseen trajectories and unseen objects? (Sec. 5.1) (2) Does our hierarchical approach help bridge the embodiment gap between human and robot hands? (Sec. 5.2) (3) Can our trained policy generalize to unseen object geometries and goal trajectories? (Sec. 5.3) (4) Can we transfer the policy from simulation to a real-world bimanual dexterous robot system? (Sec. 5.4). In this section, we first introduce our training data and baseline setups, followed by the results for each research question. Baselines. We compare our approach with the following methods and ablations: (1) Finger Joint Mapping: Match the finger joint angles between human and robot hands as demonstration and perform endto-end imitation learning [43] . (2) Fingertip Mapping: Match the fingertip positions between human and robot hands using IK as demonstration and perform end-to-end imitation learning [63] . (3) Vanilla RL: Train a PPO policy [70] to learn the motions of the arm and hand together. (4) Ours: Full implementation of our hierarchical policy learning approach. (5) Ours w. FR: Our method with an additional fingertipmatching reward during training. (6) . Ours w/o. DAL: Our method without data augmentation loop.Tasks. To validate the accuracy of the learned high-level planner, we introduce three experimental tasks: (1) Trained: Testing the policy with trained objects and goal trajectories. (2) Unseen Traj: Testing the policy conditioned on unseen goal trajectories with trained objects. (3) Unseen Obj: Testing the policy with unseen objects and trained goal trajectories.Metric. Evaluation metric is defined as the distance between the ground truth wrist pose trajectory of the hand and the wrist pose trajectory of the left and right hand output by our high-level planner. For each step, we compute translation error (TE) using Euclidean distance in centimeters and orientation error (OE) using angle difference. We report the cumulative error of the entire motion sequence in Table 1 Results. Table 1 shows that Ours performs the best in generating wrist motions, with the lowest cumulative translation and orientation error. More importantly, the performance does not drop when testing with unseen goal trajectories (Unseen Traj), which demonstrates the generalization capability of our high-level planner to novel object-centric task goals.Task. To validate the effectiveness of our framework, we first train one policy conditioned on a single trajectory in the training set for each object and test its rollout performance. Each policy is trained on a single object. multiplied by rotation angle, and 0.5 radians in joint angle). Each goal trajectory has a total of 500 action steps.Figure 4 : Experiments on the different embodiment. We study four types of the dexterous hand in three tasks from the Section 5.2.Results. Table 3 demonstrates that our hierarchical learning framework outperforms traditional hand pose matching methods (Finger Joint Mapping, Fingertip Mapping) by 50% in completion rate, indicating that our low-level RL significantly helps in bridging the embodiment gap when learning from human data. Moreover, Ours surpasses Vanilla RL by 47.3% on average, underscoring the challenge of training arm and hand actions together with RL, and emphasizing the advantage of our high-level planner for guiding the RL in high-dimensional action space. Additionally, the inclusion of human finger motions in the RL reward (Ours (w. FR)) does not yield benefits and even leads to lower performance, validating our hypothesis that the embodiment gap makes human finger motion unsuitable for training robot actions. Lastly, our data augmentation loop further brings an additional 7% improvement (Ours vs. Ours (w.o. DAL)). We further apply our method to four different types of multi-fingered dexterous hands, varying in size and degree of freedom. The results are shown in Figure 4 . Our method achieved more than 50% completion rate for all hands, demonstrating that our framework can effectively transfer human data to different robot hand embodiments.Tasks. We design four types of tasks to test the policy's generalization capability: (1) . Single Obj -Trained Traj: One policy for each object, and testing with trained goal trajectories. (2) . Single Obj -Unseen Traj: Same as prior but testing the policy conditioned on unseen goal trajectories. (3) . Multi Obj -Trained Obj: One policy trained with all objects, and testing with trained objects. ( 4). Multi Obj -Unseen Obj: Same as prior but testing the policy with unseen objects. We use the completion rate as the evaluation metric same as in Section 5.2.Results. In Table 4 , our algorithm surpassing the results of Vanilla RL on Single Obj -Unseen Traj and Multi Obj -Unseen Obj by more than 28%. This indicates that our hierarchical structure substantially improves generalization capabilities across unseen trajectories and unseen object geometries. Notably, unlike the Section 5.2, we observe an average 18% improvement in Table 3 with our DAL (Ours vs. Ours (w.o. DAL)), showcasing that the DAL greatly helps generalization. Traditional mapping methods (Finger Joint Mapping, Fingertip Mapping) and Ours (w. FR) cannot generalize to unseen trajectories and unseen object geometries due to their dependency on the finger information.Tasks. We train one policy per object conditioned on a single goal trajectory in simulation and test its rollout performance on a real-world robot system. We use the completion rate as the evaluation metric.Results. In Table 2 real-world experiments, our approach has more than a 50% completion rate improvements compared to prior methods, which barely achieve any success (< 20% completion rate) on several objects. This result showcases the ability of our approach on tackling real-world bimaual dexterous manipulation tasks. The visualization of our real-world experiments is shown in Figure 5 .  
limitations: 
 There are several limitations of our work, including our model encounters difficulties in manipulating small-size objects, the order of joints must be predefined and in our high-level planner, generalizations can only happen in the same category. More discussion about the limitations can be found in Appendix I. 
conclusion: 
 In this work, we present a hierarchical policy learning framework that effectively utilizes human hand motion data to train object-centric dexterous robot manipulation. At the core of our method is a high-level trajectory generative model trained with a large-scale human hand motion capture dataset, which synthesizes human-like wrist motions conditioned on the object goal trajectory. Guided by these wrist motions, we further trained an RL-based low-level finger controller to achieve the task goal. Our approach demonstrated superior performance across various household objects and showcased generalization capabilities to novel object geometries and goal trajectories. Moreover, the successful transfer of the learned policies from simulation to a real-world bimanual dexterous robot system underscores the practical applicability of our method in real-world scenarios. Below are the details for each augmentation. The unit of length is centimeters and the unit of angle is degrees. 
 
 Appendix: • Random the object's mesh scales with a small scale:-The scale of the width of the manipulated object ranges from 0.9 to 1.1.-The scale of the length of the manipulated object ranges from 0.9 to 1.1.-The scale of the height of the manipulated object ranges from 0.9 to 1.1.• Random the object's initial pose with a small scale:-The x-coordinate of the manipulated object ranges from -0.02 to 0.02.-The y-coordinate of the manipulated object ranges from -0.02 to 0.02.-The manipulated object's z-axis Euler degree ranges from 0 to 30.• Modify the goal trajectories of the object with waypoint interpolation:-The x-coordinate of the goal trajectories position is added by ranges from -0.02 to 0.02.-The y-coordinate of the goal trajectories position is added by ranges from -0.02 to 0.02.-The z-coordinate of the goal trajectories position is added by ranges from -0.02 to 0.02.Table .5 gives the specific information of the observation space.Denote the ĝR i , ĝT i and ĝJ i is the current 3D translation, 3D rotation and joint angle of the object respectively, the desired object 3D rotation g R i , the desired object 3D translation g T i , and the desired The reward function is defined as:EQUATIONwhere λ 1 = 20, λ 2 = 1, and λ 3 = 5.We use an exponential map in the reward function, which is an effective reward shaping technique used in the case to minimize the distance, introduced by [73, 74] . To improve the calculation efficiency, we use quaternion to represent the object orientation. The angular position difference is then computed through the dot product between the normalized goal quaternion and the current object's quaternion.Our perception setup is shown in Figure 6 . We arranged 4 identical Femto Bolt cameras around the table and face towards the object. We use FoundationPose [72] to estimate the articulated object pose.To remove the abnormal results, we compare each pose to the desired pose and remove the pose if the For the time-indexed of the goal trajectory, we randomize the sampling of input trajectories with various time gaps during training. We added an additional experiment to show that varying the time gap does not significantly affect our performance, indicating that our approach maintains generalization despite these temporal constraints. Using Coffee Maker as an example, we interpolated the goal trajectories provided by ARCTIC datasets to varying levels, and tested the completion rate of our policy on it.Origin represents the original goal trajectory from ARCTIC, and Skip 2 and Skip 1 represent skipping 2/1 poses between every two poses on the basis of origin goal trajectory. Inter 2 and Inter 1 represent insertion of 2/1 pose between every two poses according to the linear interpolation method on the basis of origin goal trajectory. Our policy generates robot actions conditioned on a sequence of object goal poses, with the time gaps defined by the distance between each consecutive goal pose. In this experiment, we randomly use time gaps of 0 to 3 units when sampling the object trajectory to test whether the policy is sensitive to these time gaps (Random Sample). The results is shown in Table 10 , the performance of the trained policy will not vary greatly. We add a baseline of the model-based method in the simulation, the result is shown in Table 11 . We use the sample-based model predictive control method that is modified from [77] as our low-level controller in our tasks. Our method outperforms the sample-based model predictive control by 46.3% on average.There are several limitations of our work. Firstly, our model encounters difficulties in manipulating small-size objects (e.g. scissors, ketchup, etc.). One potential direction is to make better use of contact information in human data. For example, [69] uses a contact graph in reward function to assist the reinforcement learning, which has the possibility of being incorporated into our framework. Secondly, we did not leverage the tactile information in the ARCTIC dataset during policy learning. In future work, we plan to equip our robot hands with tactile sensors and use contact information to train the robot to handle more contact-rich manipulation tasks. Thirdly, our method sometimes fails when the difference between the initial pose of the real-world and the goal trajectory is large. Another failure is due to the dexterous hand forms a large occlusion of the object, resulting in inaccurate pose estimation in the real-world. Fourth, The order of joints must be predefined. This limitation may not pose significant issues with single revolute joints. However, complications arise when objects feature multiple revolute joints, as the representation lacks permutation invariance across joint orders. We addressed this by focusing on category-level joint inputs consistent with URDF definitions, leveraging datasets like ShapeNet that maintain joint order consistency within categories. Future steps will involve exploring robust methods to accommodate diverse joint configurations beyond current dataset limitations. in our high-level planner, generalizations can only happen in the same category. But we still can force the model to predict action for unseen categories by sending in a trained category id and test with unseen categories. This is how we did for the generalization experiments by using the trained Box policy to manipulate the Laptop. Since there are only 11 objects in the ARCTIC dataset, it is difficult to test the capability of the generalizations across categories. In future works we will use larger 3D object dataset and human motion capture dataset to test the generalization to unseen categories. 


Decision: **Pre-rebuttal**

This paper receives three positive reviews. The reviewers acknowledge the soundness of the idea and its promise for learning bimanual manipulation from human motion data. Other mentioned merits include the novelty of the method (**u9EE**) and the clarity of writing (**u9EE**, **2mos**). Particularly, all three reviewers (**vhQ2**, **u9EE**, **2mos**) acknowledge the impressiveness of the sim-to-real demonstration.

The main mentioned weaknesses include:
- The assumption of dense object 6D pose in the demonstration and the rigidity of object representation (**vhQ2**).
- Unclear extent of generalization (**vhQ2**).
- Missing details and unclear presentation in the experiments section (**vhQ2**, **u9EE**).
- Comparison with model-based methods (**2mos**).
- Failure analysis and discussion (**2mos**).

---
**Post-rebuttal**

This paper initially received three weak accepts. All the reviewers remain positive after the rebuttal. Following the rebuttal iteration, one reviewer (**u9EE**) raised the assessment to strong accept. 

Overall the paper has delivered an impressive sim-to-real pipeline for bimanual dexterous manipulation from human mocap data. AC agrees with the assessments of the reviewers and thereby recommends accept.

Other feedback:
- After reading all the reviews/rebuttal, AC had the same question as **u9EE** of *"Table 2 shows success rates at three significant figures ... How was evaluation done in the real world (in terms of resets, success measurement)?"*, and didn't find the rebuttal response effective. "20 times of evaluation" will produce an increment interval of 5% in the completion rate and does not explain for the three significant figures here. Assuming evaluating on multiple seeds, one would need 50 seeds to produce an increment interval of 0.1% as given in Tab. 2. AC encourages the authors to further clarify this in the follow-up revision.

Typos:
- [Line 190] "Table" -> "Table."

Reviews: review: title: title: Interesting Formulations and Good Results, Need to Clarify Several Details
paper_summary: paper_summary: This paper proposes a method to learn dexterous manipulation specified by goal object trajectories. It divides the task into two stages: First, a policy generates hand wrist trajectories from given object trajectories, learned from human videos. Second, using the wrist trajectory, it uses RL to learn the finger motions. Experiments demonstrate that the proposed framework outperforms both vanilla RL and single-stage policies.
main_review: main_review: Strength:

The formulation of defining tasks according to object trajectory, while ignoring the wrist/finger trajectories, is reasonable given the embodiment gap. I believe this is an important step forward in learning more diverse tasks using sim-to-real.

Separating the learning of wrist and finger movements is interesting and shows significant performance gains. It also shows the advantage of being transferable to different hands (though only in simulation).

The demonstration of sim-to-real bimanual dexterous manipulation is impressive.

Weakness:

One limitation of this paper is the goal representation. It requires the full 6D pose of the object trajectory at each timestep, which is a strict requirement for downstream applications, as many tasks do not have such data. The representation also needs to be time-indexed, meaning that the robot needs to follow specific object velocities, and the timestep must be tuned when specifying the goal. This will limit the generalization of the formulation.

The order of joints needs to be fixed, implying that the simulation and reality need to be closely matched. This might not be problematic when there is only one revolute joint. However, it becomes problematic when objects have more than one revolute joint, in which case the representation is not permutation invariant over the joint order.

How many trials are used in the evaluation? The main text does not provide this information, and the website only displays three, which is a very small number of trials.

In the high-level planner, the policy takes the object category id c as input. It is unclear how this can generalize to different categories. Or does the generalization only happens for unseen objects within the same category?
questions: questions: When given the goal object trajectory as input, does the policy have a certain level of robustness for noisy goals? For example, if one adding position/orientation noise to the trajectory, will the policy still perform reasonably well?

How many trials are used in the real and simulation evaluations, respectively?

Does it generalize to unseen categories or unseen objects with the same trajectory? If it only generalize within the same category, this should be discussed in the limitation section.

For the high-level planner, what is the actual time (in terms of seconds) corresponding to timestep=10?

What are the difference between the proposed method and BC / BC-RNN in Table 1? I think both of them are supervised learning so I’m not sure what the authors want to show here?
limitations: limitations: 2
review_summary: review_summary: This paper presents good results with interesting formulations. I recommend acceptance but also emphasize that many details should be further clarified.
title: title: Hierarchical learning framework shows promising generalization
paper_summary: paper_summary: The paper proposes a hierarchical policy learning framework that learns high-level wrist movements from human motion with IL and lower-level finger movements with RL. The experimental results show promising generalization in objects and trajectories.
main_review: main_review: Strength:
- The paper is well written and easy to follow.
- The paper performs relatively extensive evaluations in both simulation and hardware to validate their hierarchical learning framework.
- The demonstration of sim-to-real transfer is relatively convincing.

Weakness:
- Lack of discussion on model-based techniques: The paper didn’t mention other relevant dexterous manipulation works using model-based optimization techniques such as “Global Planning for Contact-Rich Manipulation via Local Smoothing of Quasi-Dynamic Contact Models” and “Enhancing dexterity in robotic manipulation via hierarchical contact exploration” etc.
- Comparison with model-based methods: The paper focuses on comparisons with other learning-based approaches but doesn't compare against model-based optimization techniques, which are relevant in the field of dexterous manipulation.
- Failure analysis: There's limited discussion on failure modes or edge cases where the method underperforms, which could provide valuable insights for future improvements.
questions: questions: 1. Why don’t Table 1 and 2 have variances?
2. It would be great to clarify if the policy for Table 3 is trained for single or multiple objects.
3. Could the authors provide some details about why allegro hand has much lower completion rate compared to the other 3 hands?
limitations: limitations: 3
review_summary: review_summary: The core idea of leveraging human motion data to guide robot manipulation is clear and supported by substantial experimental results.
title: title: Great results on a challenging control task
paper_summary: paper_summary: The authors tackle the problem of manipulating objects using a bimanual fingered robot by decomposing it into two parts: generating rough wrist positions, and generating fine finger motions. The first subproblem is solved by imitating human-recorded trajectories, while the second is solved by training policies using RL, conditioned on the goal trajectory of the object, and on the wrist trajectory generated by the imitating controller. The method is applied to the challenging problem of tracking trajectories of articulated objects through space, using two five-fingered robots. They transfer the learned behaviours from sim to real, using a mix of explicit object state estimation and teacher-student distillation.
main_review: main_review: **Update (after rebuttal)**

During the rebuttal period, the authors improved their explanations of the Experiments section, and clarified some missing details. With these changes, I'm confident the paper should be accepted to CoRL.

**Original review**

Quality:

This paper presents some excellent results in simulation, with decent transfer to real, on complicated manipulation tasks.
While I have confidence that the system works (based on the shared videos), the Experiments section lacks a lot of detail and is hard to understand.

Clarity:

With the exception of the Experiments section (expanded below), the paper is well-written and clear. The method is motivated and described well.

Originality:

The idea of separating large arm movements from fine low level control has been tried before, as well as the idea of imitating human data for the large arm movements.
The most similar work I could find was (Sudeep Dasari; Abhinav Gupta; Vikash Kumar, 2023), where human data was used to generate a pre-grasp position for the arm, and RL was used to control the arm and fingers from then on.
This paper introduces a couple of advances (bimanual manipulation, real robot experiments, full use of recorded wrist trajectories instead of just a pre-grasp) that make it sufficiently interesting.

Significance:

This work shows successful results on real-robot manipulation of articulated bodies using bimanual multifingered robots, and will certainly inspire future works in this domain.

Strengths:

The problem tackled is challenging, and the approach is explained clearly. The real robot results are impressive.

Weaknesses:

The Experiments section is hard to parse and lacks a lot of details, to explain what numbers are being presented. While reading it, I came up with the following questions:

* In section 5.1, when evaluating different high level planners, is the low-level controller frozen or trained separately for each high level planner?

* In table 1, orientation errors are reported as Frobenius norm, which is hard to conceptualize. Since in other parts of the paper you talk about angle difference (e.g the completion criterion on line 200), could you convert those numbers to angle difference too?

* In section 5.2, the definition of the completion rate seems very lenient. In particular, a 0.5 radian (28 degree) error seems impractical for any application, particularly for large objects. Given a few of the results saturate at 100%, reducing this threshold seems sensible. Either choosing a small number (e.g. 0.1 radian), or scaling it by the object's longest dimension (e.g. L * angle < 2.5cm) would be fine.

* In section 5.2, why doesn't the completion criterion include a mention of the articulation of the object?

* I couldn't find out what the $\pm$ figures in table 3 are. Are they confidence intervals or standard deviations? Why do some results with similar success probabilities have very large differences in the confidence interval? Were some evaluations run more times than others? Please include this information in the appendix. For estimating confidence intervals for a binomial proportion, consider using something other than the normal approximation (https://en.wikipedia.org/wiki/Binomial_proportion_confidence_interval).

* Table 2 shows success rates at three significant figures for real world evaluations, but I wasn't able to find the number of executed trajectories in those evaluations. Could this data be included in the appendix? Probably, the number of significant figures should be reduced. How was evaluation done in the real world (in terms of resets, success measurement)?

* In the supplementary video, some sim policies have significant vibrations in the arms that aren't present on the real robot. Is this a result of sim instability or of the low-pass filter mentioned in section 4.4?
questions: questions: * Please provide all the details required to understand the numbers presented in the experimental evaluations.
* When referring to (Sudeep Dasari; Abhinav Gupta; Vikash Kumar, 2023), please explicitly address the differences between your approach and theirs, as they are similar (and got similar sim results).
* Please add a citation to an early work showing independent control of the fingers and arm, e.g. http://yutingye.info/SIG12.html.
limitations: limitations: 3
review_summary: review_summary: This is a strong paper with impressive performance on a real complex embodiment.