TieBot: Learning to Knot a Tie from Visual Demonstration through a Real-to-Sim-to-Real Approach

Authors: weikun  peng
jun  lv
yuwei  zeng
haonan  chen
siheng  zhao
jichen  sun
cewu  lu
lin  shao
Venue: robot-learning_org_CoRL_2024_Conference
Published: 2024-06-04 19:12:30

Abstract: The tie-knotting task is highly challenging due to the tie's high deformation and long-horizon manipulation actions. This work presents TieBot, a Real-to-Sim-to-Real learning from visual demonstration system for the robots to learn to knot a tie. We introduce the Hierarchical Feature Matching approach to estimate a sequence of tie's meshes from the demonstration video. With these estimated meshes used as subgoals, we first learn a teacher policy using privileged information. Then, we learn a student policy with point cloud observation by imitating teacher policy. Lastly, our pipeline applies learned policy to real-world execution. We demonstrate the effectiveness of TieBot in simulation and the real world. In the real-world experiment, a dual-arm robot successfully knots a tie, achieving 50% success rate among 10 trials. Videos can be found on https://tiebots.github.io/.

Main paper: 
 introduction: 
 Learning cloth manipulation holds great utility across a wide range of applications. One intriguing domain is robotic tie knotting. Service robots must be adept at tasks like aiding the elderly or individuals with disabilities in dressing for certain social events. Teaching robots to knot ties, as a special case of cloth manipulation, typically pushes the limits of robotic cloth manipulation. This offers valuable insights for tie knotting and the broader field of robotic cloth manipulation.Cloth manipulation presents challenges for robots due to its high-dimensional state and complex dynamics. Extracting and modeling state information are difficult problems. In contrast, humans have accumulated extensive knowledge about cloth manipulation. These priors make learning from demonstration (LfD) a promising direction. LfD empowers a robot to acquire a policy from expert demonstrations, significantly reducing the need to design task-specific reward functions manually. Consequently, LfD stands as a potent and efficient framework for instructing robots in the execution of complex skills. However, existing LfD methods struggle with tie-knotting tasks. Kinesthetic demonstration or teleoperation suffers from the complexity of tie-knotting tasks. Tie-knotting tasks require bi-manual operations, placing high demand on human operators' skills and equipment. For instance, Zhang et. al use VR headsets for teleoperation [1] . Thus, simple behavior cloning may be significantly laborintensive. Learning from visual demonstration is usually an easier approach in terms of collecting † Correspondence to peng.weikun@u.nus.edu and linshao@nus.edu.sg 8th Conference on Robot Learning (CoRL 2024), Munich, Germany. demonstration data. But this approach also leads to embodiment gaps. Therefore, researchers attempt to find some object-centric representations that robots can utilize to generate correct actions, overcoming embodiment gaps. Several methods attempt to learn a general visual representation of some simple pick-place skills via large-scale pre-training on actionless videos [2, 3, 4] . These works present strong generalizations on the learned visual representations, but none of them shows the ability to learn dexterous manipulation skills that can knot a tie. Other methods such as [5, 6, 7, 8, 9] try to leverage object trajectories or keypoints as representations to guide the policy learning. Such representations are indeed sufficient to describe simple object motions but fail to capture the tie's complex topology and subtle dynamics.Learn@Sim Sim2Real Manipulation Figure 1 : Our proposed TieBot performs a tieknotting task. We leverage cloth simulation to recover the cloth's state from human demonstration and learn a goal-condition policy to accomplish the tie-knotting task.Compared to the existing LfD work mentioned in the previous paragraph, our insight is that mesh is the most suitable representation for tie-knotting tasks and other complex cloth manipulation tasks. It captures accurate geometric structures and physics properties of the tie, which is crucial for tie-knotting tasks. It also disentangles irrelevant information in the visual demonstrations, such as environment background, object colors, and so forth, enabling the learned policy to apply to different test settings. Therefore, inspired by [10] , we propose a Real-to-Sim-to-Real LfD framework. First, we propose a Hierarchical Feature Matching method to iteratively estimate the tie's meshes with cloth simulation from the demonstrated video. We use a cloth simulator called Dif-fClothAI [11] that supports intersection-free contact for cloth to maintain the tie's topological structure during the estimation process. These estimated meshes from the demonstrated video are then used as subgoals. To learn where to grasp the tie and where to pull the tie from point clouds observations in simulation, we adopt a teacher-student training paradigm similar to [12] . Lastly, our pipeline executes learned policy in real-world settings.In summary, we make the following contributions: 1) We introduced a systematic LfD framework for a dual-arm robot to learn to knot to tie.2) We proposed a Hierarchical Feature Matching approach to estimate the tie's mesh with high deformation from the demonstrated RGB-D video using cloth simulation. 3) With estimated meshes as subgoals, we presented a teacher-student training paradigm to learn grasping points and placing points from point cloud observations in simulation. 4) We conduct experiments in simulation and the real-world to demonstrate the effectiveness and advances of our pipeline. To the best of our knowledge, this work is the first effort to develop a robotic system that integrates perception, modeling, and robot learning to train robots to tackle the task of tie-knotting-a particularly complex and underexplored area of cloth manipulation. 
cloth manipulation: 
 Previous work mainly addresses short-horizon cloth manipulation tasks that only involve simple pick-place actions. There are several approaches to learning cloth manipulation skills. One approach is using model-free RL or learned dynamics model to learn cloth unfolding, rope rearranging, and dressing assistance tasks on raw sensor input [13, 14, 15, 16, 17] . Other approaches will collect and annotate data from images [18, 19] or generate demonstration trajectories in simulation [20] to learn policy. Because of the short-horizon and simple actions features of tasks, it's also possible to infer correct actions from some visual representations, such as flow between current observation and target images [21] .In contrast, tie-knotting tasks require flipping or rotating a part of the tie, which makes it difficult to annotate robot actions or design action primitives. Therefore, collecting and annotating robot actions on observations is infeasible. It's also difficult to generate demonstrations or directly apply RL in simulation since the trajectories of tie-knotting tasks are much longer and the possible state space is much larger. Thus, in this work, we choose to learn skills from human demonstration.One line of research explores pre-training neural representations from actionless videos [22, 23, 2, 24, 3, 4, 25] . This approach aims at learning general representations for different actions, whereas none of them shows the ability to learn dexterous manipulation skills that can knot a tie. Another line of research attempts to learn from visual priors extracted from visual demonstrations, such as object trajectories [5, 26] , hand poses [27, 28] , keypoints positions [29, 6] , graph relations [7] , or affordances [30] . The third approach is to learn a video or trajectory prediction model to guide policy learning [31, 32, 33, 34, 8] . These approaches require in-domain demonstrations, placing restrictions on visual demonstrations. Moreover, the prediction model may suffer from the long-horizon feature of tie-knotting tasks. ORION is the most closely related work, which builds a graph representation from object motions that can generalize across diverse test environments [35] . However, simple graph representations cannot capture the tie's complex topology and subtle dynamics during the tie-knotting process.Consequently, we propose explicitly modeling the demonstration as a sequence of meshes. Mesh can accurately describe the tie's structure and dynamics, which is crucial to learning correct robot actions and generalizing them to different test scenarios.One cloth state estimation method directly predicts cloth states using deep learning [36, 37, 38, 39] . Non-rigid point cloud registration methods such as coherent point drifting are also applied for linear deformable object tracking [40, 41, 42] . However, purely vision-based methods do not guarantee correct cloth topology due to the lack of physics prior. Huang et al. propose a method to reconstruct and track cloth state with a dynamics model [43] . However, this method requires known actions, which cannot be accessed easily from human demonstration sometimes. Lv et al. use differentiable rendering to estimate the state of linear deformable objects [10] . However, tie is a 2D deformable object.Therefore, we propose a Hierarchical Feature Matching method to iteratively estimate the tie mesh in the demonstration video with cloth simulation. Cloth simulation provides important physics prior for state estimation, such as non-penetration, which is crucial for maintaining correct topology. 
technical approach: 
 This work presents a Real-to-Sim-to-Real LfD framework called TieBot to guide a dual-arm robot shown in Fig. 1 to knot the tie from an RGB-D demonstration video. An overview of our proposed method is in Fig. 2 . We first describe the procedure to estimate the tie's mesh sequences from the demonstrated video (Sec. 3.1). Using the tie's mesh sequences as subgoals, we introduce a pipeline to generate robot actions to manipulate the tie, using teacher-student training paradigm (Sec. 3.2). To better estimate the tie's mesh, we propose to integrate cloth simulation into our pipeline, which provides important physical prior such as non-penetration in the estimation process. We segment the tie in the demonstrated RGB-D video using Track-Anything [44] and transform the associated segmented depth images into point clouds. Meanwhile, a tie's mesh is loaded into the DiffClothAI. At time step t, we use the tie mesh's vertices denoted as Y S t to describe the tie's shape. From the RGB images and segmented point clouds denoted as {I D t } and {X D t }, Real2Sim pipeline estimates tie's mesh sequences {Y S t } with simulation. The pipeline manually aligns the mesh with the initial frame. We assume the initial mesh fully overlaps with the initial point cloud. are aligned and there are correspondences between X D t-1 and X D t , we can build the correspondences from the tie's mesh to the next demonstrated point cloud X D t and move the tie's vertices to align Y S t towards X D t . Here we adopt an off-the-shelf feature matching model called LoFTR [45] to build up correspondences between two RGB images I t-1 and I t as shown in Fig 3 . Typically, LoFTR can provide more than a hundred reliable correspondences between two images, which cover almost every visible part of the tie. From the correspondences between I t-1 and I t , we can find the feature points on X D t-1 and their corresponding feature points on X D t . Then, to control the mesh in Diff-ClothAI to align it with X D t , we need to define several vertices on the mesh as control vertices V. Since Y S t-1 aligns well with X D t-1 , we map the feature points on X D t-1 to nearest vertices on Y S t-1 . These vertices are assigned as control vertices V. Finally, we control V to move to the positions of feature points on X D t to align Y S t towards X D t in DiffClothAI. However, vanilla local feature matching cannot create correspondences in occluded regions, which is common in tie-knotting tasks. We lose the motion information due to occlusion, and the estimation will deviate. Therefore, we propose to add global keypoints information to amend this pipeline.Keypoints Detection. Keypoints detection can directly build correspondences between mesh vertices and the point cloud. Thus, it will not be affected by occlusion. We define five keypoints along the tie's surface and the corresponding five key vertices on the mesh, shown in Fig. 4 . For each keypoint as the origin, we define the local frame as follows. The z direction is the surface normal from the tie's positive side to the negative side. The x direction is the direction of the tie's middle skeleton. The y direction is derived using the right-hand rule. These five keypoints, in a predefined order, play the role of the skeleton definition. Then, we train Pointnet++ [46] to predict the keypoints and associated local frames on the demonstrated point clouds. However, the highdimensional state makes it challenging to generate sufficient training data to cover all the states encountered in the knotting procedure. A successful tie-knotting trajectory occupies only a small portion of the whole state space of the tie. Thus, uniformly applying random actions on the initial tie's mesh in the simulation to produce training data fails to cover these states. In contrast, we generate training data based on the current estimated mesh. When we detect the chamfer distance between Y S t and X D t is larger than a threshold, we backtrack to the previous time step, gather the tie's shape Y S t-1 and apply random actions to the tie's mesh at t -1 in the simulation to generate annotated training data and train the keypoints prediction network.Hierarchical Feature Matching (HFM). Finally, we combine them as Hierarchical Feature Matching (HFM) for state estimation. Control vertices assigned in local feature matching and key vertices will be used together to pull the mesh to target positions specified by local feature matching and keypoints detection. Local feature matching provides detailed motion of vertices, while global keypoints indicate a global tie's structure. This global structural information ensures the estimation won't deviate too much, alleviating the shortcomings of the local feature matching method. We use this method to estimate the tie's meshes from demonstration and output a sequence of meshes {Y G t }. The next part will use these meshes as subgoals to guide robot action generation.Our pipeline begins to sequentially generate feasible robotic actions in the simulation to guide the tie {Y S t } towards these subgoals. Since the tie-knotting task is a long-horizon task with multiple grasp and pull actions, we aim to learn where to grasp the tie and where to pull the tie.For where to pull, we apply a simple strategy. Once we identify the grasping vertices, we pull these vertices to the positions of those vertices on the subgoal. For where to grasp, we adopt a similar teacher-student training paradigm in [12] to ease policy learning. Directly learning from high dimensional observations such as point cloud is data-inefficient because the policy needs to simultaneously learn which features to extract from visual observations and what the high-rewarding actions are. On the contrary, learning a policy via RL from sufficient state information would be much easier, as suggested by [12] . Therefore, we first use privileged information to learn a teacher policy, and then train a student policy imitating teacher policy with point clouds as observations. Teacher Policy. We first learn a teacher policy to select proper grasping points using privileged information. The state s contains the previous tie's vertices positions and the point-wise displacement for each tie vertices to the subgoal. The action a is one or two grasping vertices of all the tie mesh's vertices. The reward function R is defined in equation 1.Note that we specify the action space as the discrete space (vertex index of the tie). Although there are multiple 6D poses of the robotic grippers to grasp one vertex position of the mesh, the learned policy still reflects the overall grasping quality of these 6D poses associated with one vertex. In the engineering practice, we record each grasping pose offline so that once we figure out the grasping vertices on the tie's mesh at each timestep, we can automatically produce the feasible grasping poses concerning specific hardware platforms using inverse kinematics.R(s, a) =    C 1 , if knotting-tie succeeds -C 2 ,if fails to reach any subgoal along the trajectoryEQUATIONFor the reward function, here C 1 , C 2 , C 3 are constant positive values. X S t is the result tie mesh. The failure to reach the subgoal is due to the distance ∥Y S t -Y G t ∥ is larger than a given threshold, the tie could not be pulled close to the subgoal by grasping on the wrong selected vertex; otherwise, it will return C 3 -∥Y S t -Y G t ∥ for intermediate steps or C 1 for the final step. Student Policy. To learn actions from point clouds, we train a student policy to imitate teacher policy. We add some perturbations to the size and positions of the mesh and update the associated trajectories accordingly to generate training data in the simulation. We render point clouds from meshes in PyBullet [47] as the input of our policy network π sim and output the grasping points and placing points positions. We use Pointnet++ [46] as the policy network and train it in a supervised learning manner. 
experiments: 
 In this section, we introduce our experimental setup and conduct quantitative and qualitative evaluations to demonstrate the effectiveness of our approach. Our experiments focus on answering the following questions.• How do our pipeline and baseline methods perform on tie-knotting task?• Can our pipeline apply to other cloth manipulation tasks?• How does HFM compare to other cloth state estimation methods?• Can our HFM apply to other cloth manipulation tasks?Considering the complexity of the entire system, we provide additional experiment results, along with detailed explanations of submodules, in the supplementary materials and website.We first evaluate the whole pipeline of TieBot and a baseline method in a tie-knotting task. We estimate a sequence of meshes from one human demonstration video. Then, we divide the whole trajectory into 6 parts with 6 subgoals. Our teacher policy learns to select proper grasping points using PPO [48] , and student policy imitates the teacher policy to infer grasping points and placing points from the point cloud. We evaluate TieBot and the baseline method 10 times for each of the two different ties in DiffClothAI and evaluate TieBot on two real ties with a dual-arm robot.To illustrate our pipeline applies to other cloth manipulation tasks besides the tie-knotting task, we conduct experiments on the towel-folding task in the real setting. The towel-folding process is shown in the last row of Fig. 7 . The action is the 3D offset of the grasping vertices. For towel-folding task, we collect 100 human demonstrations and 10 robot demonstrations in the real world. We train the ATM to predict the displacement of the two robot arms' end effectors in cartesian space.Metrics. We compare the success rate between our pipeline and ATM [8] . For the tie-knotting task, in simulation experiments, if the distance of the final tie's mesh to the target tie's mesh is smaller than a threshold, we consider it a success. In real-world experiments, if the little end of the tie is inserted Experiment Result. We test TieBot on two real ties that differ in materials: one is softer, and the other one is harder. We tested each of them 10 times. We also test TieBot and ATM on two ties with different sizes in simulation 10 times for each. The quantitative results are shown in Tab. 1, and qualitative results of TieBot are shown in Fig. 5 . This comparison suggests that object trajectories are insufficient to represent subtle dynamics and topology of the tie in tie-knotting tasks. ATM quickly deviates from the correct trajectory since it cannot capture the subtle dynamics of the tie. Therefore, it fails to achieve even one subgoal. Hence, explicitly modeling the tie in meshes is necessary. For qualitative evaluation, in Fig. 5 , we can see that although the tie in the demonstration video, the mesh in the simulation, and the tie used for real robot manipulation are different, our pipeline can overcome these gaps and learn feasible robot policy. For the towel-folding task, we find that ATM can learn the first folding action but struggle with learning the latter action.Real2Sim is the most important part of our pipeline. Without accurate state estimation, particularly estimating the correct topology for the tie, it's impossible to learn a feasible policy to accomplish the task. To illustrate the importance of different components of HFM and its performance against other cloth state estimation methods, we design three experiments in simulation to test baseline methods and the ablation versions of HFM.Coherent Point Drift. Coherent Point Drift (CPD) [49] is a non-rigid point cloud registration algorithm. We employ the CPD to predict the target positions of the mesh vertices in the target point cloud and directly align the mesh to the target positions. Experiment Result. The qualitative results are shown in Fig. 6 . We can find that either CPD or ablated versions of HFM cannot estimate the target mesh correctly among these three experiments.We also compute the L2 Distance between the vertices of the target mesh and estimated mesh as a quantitative evaluation shown in Tab. 2. It also suggests that the performance will degrade if we cancel some parts of HFM, while CPD deviates a lot from the correct states.We demonstrate that HFM can be applied to other cloth manipulation tasks. One is a different way to knot a tie. The other one is to fold a towel. We visualize the estimation results in Fig. 7 . The results show that HFM can be applied to different cloth manipulation tasks.Figure 7 : The visualization of a different way to knot a tie and the towel folding. The first row is the human demonstration of tie-knotting, the second row is the estimated states in simulation, and the third row is towel folding. We show the manipulation action in yellow dots and arrows. 
conclusion: 
 This paper presents TieBot, a novel system designed to teach robots to perform the tie-knotting task through a Real-to-Sim-to-Real learning approach. The framework leverages visual demonstrations and integrates a mesh of the tie to capture its complex structure and subtle dynamics, which are essential for effective manipulation. Through the proposed Hierarchical Feature Matching method, we estimate tie meshes from demonstration videos and use them as subgoals for policy learning in simulation. Our teacher-student training paradigm enables robots to learn from point cloud observations and execute the learned policy in real-world settings. TieBot demonstrates promising results both in simulation and real-world experiments, achieving a 50% success rate for tie-knotting. Additionally, we showcase its potential for other cloth manipulation tasks, such as towel folding, indicating the broader applicability of our approach. This work marks a significant step forward in robotic cloth manipulation, particularly for long-horizon, complex tasks such as tie-knotting.Nonetheless, our pipeline has some limitations. First, our Real2Sim module requires training keypoints detection models iteratively, which is computationally intensive. Second, due to the hardware limits, the last step in the real-world experiments shown in Fig. 5 is hardcode action. Better video tracking methods and more dexterous robot arms may alleviate these issues. To annotate training data, we first compute the geodesic distance of all points in the point cloud to keypoints. Then, we convert the geodesic distance to probability using equation 1. d is the geodesic distance, σ is a hyperparameter, p is the probability to evaluate how likely the point is to be a keypoint. In our experiments, we use σ = 0.15. Thus, if a point is close to one of five keypoints, the probability corresponding to that keypoint will be high.EQUATION2) training details Different from the original pointnet++ semantic segmentation model [46] , we change the last layer to sigmoid. 3) inference details Our model takes a point cloud as input and outputs a probability matrix P ∈ (0, 1) N ×5 , N is the number of points in the point cloud. Each entry P i,j represents the probability of point i to be keypoint j. To decode the predicted keypoints positions, we first select points with the top 5% probability as inlier for each column of P . Then, we assign other points' probability to zero and normalize the probability for each column of P . Now we get the normalized probability distribution of each keypoint, denoted as P . Finally, we compute the average positions of all points weighted by normalized probability, x k = N i=1 Pi,k • x i . This is the final prediction for keypoint positions.  
 
 Appendix: Grasp Actions For the grasp action, we begin by sampling a set of vertices from the mesh, which serve as grasping vertex candidates. The grasp action is then defined as selecting different combinations of these vertices. This selection process is treated as a discrete action space, where the policy chooses the optimal vertices for grasping based on the current state of the tie.The pull action is defined as the target position for the robot's gripper. Once the policy has selected the grasping vertices at a particular step, the corresponding goal positions are determined by the positions of these vertices on the subgoal mesh. This approach means that the pull action is not learned directly; instead, it is derived automatically based on the selected grasping points and the configuration of the subgoal mesh, leveraging the differentiable cloth simulation.We model the grasping point selection as MDP and use model-free RL to learn the proper grasping point. To simplify the problem, we sample 40 vertices on the middle line of the mesh model as our candidates. vertices directly connected to these candidates in left, right, up, and down are defined as their neighbors.In practice, we evenly sample 40 vertices on the middle line of the mesh model as our candidates. The state s is a 40 × 6 matrix. The action a is a 820 × 1 one-hot vector, which contains grasping one vertex (40) and two vertices(40 × 39/2 = 780).To learn to select grasping points, we use PPO implemented in stable-baseline3 [52] To learn student policy, we first execute the teacher policy multiple times within the DiffClothAI simulation environment. During each run, the teacher policy uses the current mesh and the subgoal mesh to predict the selected grasping vertices on the current mesh, as well as the corresponding placing points. Through these simulations, we generate a dataset comprising 3,000 mesh-graspingplacing pairs, which serve as the foundation for training the student policy.To simulate real-world conditions, we load all the generated meshes into PyBullet and use the same camera intrinsics as those used in our real-world setup to render point clouds. This ensures that the point clouds accurately reflect the observations a robot would receive in practice.The student policy is then trained using supervised learning, where the input is the rendered point cloud and the output is the predicted grasping points and placing points. The grasping and placing points produced by the teacher policy serve as the ground truth labels for this training process. The student policy learns to map point cloud observations to the appropriate actions (grasping and placing points), effectively mimicking the decisions made by the teacher policy based on the point cloud data alone.The training details are the same as training keypoints prediction, only changing the number of keypoints from 5 to 2. We follow the same training hyperparameters as grasping point prediction for placing point position detection, only changing the pointnet++ semantic segmentation model to the classification model.We first illustrate example outputs of ATM baseline on two different ties in Fig. B.6. We can see that without explicit mesh modeling, ATM will quickly deviate from correct trajectories.  


Decision: The paper introduced a method for tie knotting from video demonstration containing three phases. The first phase is to estimate mesh sequences from demonstration videos. The second phase uses the estimated mesh sequences as subgoals to learn a policy in simulation to act from the point cloud. Finally, the third phase learns a residual policy to aid the simulation policy when deployed in the real.
The paper deals with the very challenging manipulation tie-knotting problem, is well-written, and provides thorough experimentation and analysis especially its tracking ability.

The reviewers had originally important comments and concerns on the specificity of the approach and its components to the tie-knotting problem, the limitations in terms of its performance, and the limited scope of the experiments on generalization for the task and for the object (tie).

While the authors well-addressed the comments of the reviewers, I agree that more experimental evaluation with the real robot should be included in a robotics paper. Therefore I recommend the paper to be accepted for poster presentation.

Reviews: review: title: title: Encouraging results in the challenging task of tie knotting from human demonstration video, but need more experimental results
paper_summary: paper_summary: The paper introduced a method for tie knotting from video demonstration containing of three phases. The first phase is to estimate mesh sequences from demonstration videos. The second phase use the estimated mesh sequences as subgoals to learn a policy in simulation to act from the point cloud. Finally, the third phase learn a residual policy to aid the simulation policy when deployed in the real.
main_review: main_review: ### Clarity:
- The clarity can be improved. I listed some below, but there might be more
- Table 2 has overlapping texts
- Citations should be added where necessary (e.g., CPD)
- There are places in the text spaces should be added (e.g., the title of 4.2, 3.1.3, mitigate Sim2Real gaps, places in appendix)
- Some consistency of text capitalization (e.g., title of 4.1)
- I suggest using a clearer notation in 3.1 so that readers can know which notations are the images, point cloud, meshes, and come from simulation or demonstrations. The current notation X refers to both point clouds and mesh sequences and is quite confusing
- The authors should talk about ATM first before using it in the Metrics section in 4.1
- In Figure 6, the authors should use a different color for the red cross sign to make it easier to see

### Originality:
- The paper has a certain level of novelty

### Significance:
- The paper tries to tackle a very challenging robotics problem with a proposed agent that can succeed half of the time in simulation

### Weaknesses:
- The results are encouraging for such a challenging task but still limited both in terms of the performance and the scope of the experiments (only one baseline is compared with and, two types of ties are used for experiments, limited ablation studies are provided)
- There are several manual steps in the proposed approach (e.g., manual key points definition), so it is unclear about the applicability of the method in other settings (there is some discussion in 4.3 about applying HFM to other cloth manipulation tasks but seems insufficient)
questions: questions: - The paper stated the student policy is trained using supervised learning; more details should be provided
- In Table 1, which result comes from simulation and which comes from real hardware?
- How do the authors connect PyBullet with DiffClothAI to train the student policy?
- In Table 2, does the result come from averaging all the meshes or just a single one, and what is the unit?
- It seems that it is pretty easy to experiment with the method for towel folding in the hardware; the authors are encouraged to add such an experiment instead of just showing the performance of HFM. This task can have more baselines to see the performance of the proposed method to state-of-the-art baselines
limitations: limitations: 3
review_summary: review_summary: The paper introduced an interesting method for learning tie knotting from human demonstration videos. The method has certain level of successes and advancement but more experiments should be performed and the paper's clarity should be improved.
title: title: Review of TieBot
paper_summary: paper_summary: This work proposes a system for learning to knot a tie by using human video to generate subgoals for supervising policy training in simulation which is then transferred onto real hardware.
main_review: main_review: TieBot is a system for learning to tie knots via real to sim to real approach. The system involves first estimating subgoals of tying a knot from human video demonstrations by building correspondences with a mesh of the tie. These subgoals are then transferred to simulation and used to train a policy operating over privileged information, which is then distilled into a vision-based policy that is transferred to the real world. 

The work is largely well written with a few clarification issues that I note in the question. The idea is simple and straightforward yet novel, using human video demonstrations to bootstrap simulation training and transfer back to the real world. The results are not particularly strong, the method only works half the time in the real world, but it is a fairly challenging task to operate from visual input so it is still impressive. 

Strengths:
* evaluation on a complex manipulation task with first of its kind results
* thorough experimentation and analysis
* well written 
* intriguing method for using human video to bootstrap simulation training, but also largely problem specific

Weaknesses:
* limited (though impressive) real world evaluation
* no real generalization, as far as I can tell it is trained and tested on the same tie
* method seems very specific to knot tying
questions: questions: 1. Do you need to re-train a policy for each tie? Is the whole pipeline tie specific?
2. How much randomization is present in real world trials? I found "We test our real-world policy 10 times. Each time the initial positions of the tie are slightly perturbed about 5cm. The final success rate is 50%" but could not find a more detailed description in the paper or supplement. 
3. What is the compute required to train the policies, how long does training take? 
4. How are the grasp/pull actions defined, can the methods section be updated to make this much more clear?
5. 74-75: To the best of our knowledge, this work is the first effort to develop a robot system, integrating perception, modeling, and robot learning. This statement is clearly not true, perhaps the authors would like to amend it to a statement regarding knot tying? 
6. Is the human arm segmented out in the intermediate steps of the video?
7. Can this method be applied to any other domains?
limitations: limitations: 3
review_summary: review_summary: I recommend accepting, I think this work has sufficient novelty and evaluation to meet the criteria for acceptance, though it would be helpful to clarify some of the minor concerns and comments.
title: title: Review
paper_summary: paper_summary: The paper proposes Hierarchical Feature Matching, combining correspondence-based local feature matching from RGB and learning to predict global keypoints from pointclouds. This novel state estimation method allows for real2sim transfer of the cloth’s state from a single human demonstration. The resulting sequence of meshes then act as subgoals for policy learning in simulation. To enable sim2real transfer, the authors utilize student-teacher distillation in simulation and residual policy learning in the real world.
main_review: main_review: Pro
1. Impressive tracking ability of the presented real2sim approach compared to video tracking under occlusions and hard-to-model cloth.
2. Extensive evaluation of the tracking approach (different initial poses of the tie, cloth material, and towel folding).
3. Real-world tie knotting experiment with a bi-manual robot.
 
Con
1. Lack of detail in some aspects of the pipeline (see Questions For Rebuttal).
2. Missing robot experiments for the towel folding experiment.
3. It is unclear how much the residual policy learning contributes and how it was implemented.

Minor comments:
1. Line 74-75: “To the best of our knowledge, this work is the first effort to develop a robot system, integrating perception, modeling, and robot learning” This claim feels very general and in its current form like an overstatement. For instance, here are three real2sim works that also perceive, model the real world, and learn a downstream policy:
- Reconciling reality through simulation: A real-to-sim-to-real approach for robust manipulation. Torne et al. RSS 2024
- URDFormer: A Pipeline for Constructing Articulated Simulation Environments from Real-World Images. Chen et al. RSS 2024
- ASID: Active Exploration for System Identification in Robotic Manipulation. Memmel et al. ICLR 2024
questions: questions: 1. The reward function design in Equation 1 mentions variables C_1, C_2, C_3 but the experiment section never provides the concrete values being used. How much does the choice of those matter and how much tuning did you have to do?

2. The sim2real stage includes training a residual policy. Is this policy trained in the real world? Providing additional details, e.g., how many episodes were needed and how performance evolves over time would help better understand the sim2real gap.

3. The success rates in Table 1 show that knotting a tie from a single human demonstration video is still challenging. How do the failure cases look like especially w.r.t. to the different properties of the tie (unseen, softer, harder)?

4. Would collecting more than one demonstration benefit the state estimation and real2sim?

5. Line 296-297: “[The] pipeline still requires manually setting the initial state of the tie at the beginning of the Real2Sim stage.” How does this process look like and can it be automated?
limitations: limitations: 3
review_summary: review_summary: The presented method shows impressive tracking ability in a real2sim setting as well as when executed on a real-world robotic platform. Minor improvements like real-world experiments on a different setup (e.g. towel folding instead of tie knotting) and adding ablations for the residual policy learning would further improve the paper.