Title: DexGarmentLab: Dexterous Garment Manipulation Environment with Generalizable Policy
Venue: NeurIPS 2025
Accepted: yes
Keywords: Garment Manipulation; Datasets and Benckmarks; Dexterous Hands Manipulation; Automated Data Collection; Imitation Learning;
Abstract: Garment manipulation is a critical challenge due to the diversity in garment categories, geometries, and deformations. Despite this, humans can effortlessly handle garments, thanks to the dexterity of our hands. However, existing research in the field has struggled to replicate this level of dexterity, primarily hindered by the lack of realistic simulations of dexterous garment manipulation. Therefore, we propose DexGarmentLab, the first environment specifically designed for dexterous (especially bimanual) garment manipulation, which features large-scale high-quality 3D assets for 15 task scenarios, and refines simulation techniques tailored for garment modeling to reduce the sim-to-real gap. Previous data collection typically relies on teleoperation or training expert reinforcement learning (RL) policies, which are labor-intensive and inefficient. In this paper, we leverage garment structural correspondence to automatically generate a dataset with diverse trajectories using only a single expert demonstration, significantly reducing manual intervention. However, even extensive demonstrations cannot cover the infinite states of garments, which necessitates the exploration of new algorithms. To improve generalization across diverse garment shapes and deformations, we propose a Hierarchical gArment-manipuLation pOlicy (HALO). It first identifies transferable affordance points to accurately locate the manipulation area, then generates generalizable trajectories to complete the task. Through extensive experiments and detailed analysis of our method and baseline, we demonstrate that HALO consistently outperforms existing methods, successfully generalizing to previously unseen instances even with significant variations in shape and deformation where others fail. Our project page is available at: https://wayrise.github.io/DexGarmentLab/.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper introduces DexGarmentLab, a comprehensive framework for bimanual dexterous garment manipulation. Contributions include: (1) A new simulation environment built on Isaac Sim, featuring over 2,500 garment assets from ClothesNet across 15 task scenarios; (2) an automated data collection pipeline that is able to generate datasets of diverse trajectories from a single expert demonstration using a Garment Affordance Model (GAM) to identify manipulation points on new garments; (3) a hierarchical policy HALO which uses GAM to generate manipulation trajectories conditioned on visual and state information. The approach is validated in both simulation and in the real world.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n- [Quality]: The DexGamentLab ecosystem is a complete package with a feature-rich simulation environment, a large-scale asset library, an automated data collection methodology, and a policy. 14 simulation tasks and 4 real robot tasks further strengthen its statistical significance.\n- [Clarity]: The paper is generally well-organized and easy to follow, considering the amount of work done.\n- [Significance]: The framework provides the first bimanual dexterous garment manipulation environment at this scale. Automated data collection reduces reliance on teleoperation. The development of a high-quality, open-source environment and a promising baseline policy is a valuable resource for further research in this area.\n- [Originality]: The integration of structural correspondence (GAM) with diffusion-based trajectory generation (SADP) for category-level generalization is novel.\n\nWeaknesses:\nThe paper's primary weakness stems from a significant mismatch between the scope of its claimed contributions and the focus of its empirical evaluation. The authors frame the work as having three main contributions: the DexGarmentLab environment, the automated data collection pipeline, and the HALO policy. However, the experimental section is almost exclusively dedicated to evaluating the performance of the HALO policy against baselines and ablations. The other two pillars of the contribution lack corresponding empirical validation, creating a structural imbalance in the paper.\n- [Lack of Experimental Validation for DexGarmentLab]: While the qualitative comparisons in Table 1 and the descriptive improvements to the physics model are informative, they are not supported by experiments. While a key claim is that these new physical parameters reduce the sim-to-real gap and provide more realistic interactions, there is no experiment designed to quantify this benefit. For instance, an analysis showing that training a policy is more sample-efficient or faster in terms of wall-clock time compared to other environments or using the AttachmentBlock method would be necessary to empirically substantiate the environment's value as a standalone scientific contribution.\n- [Lack of Quantitative Analysis for the Data Collection Pipeline]: Similarly, the automated data collection pipeline is also presented as a major contribution. However, the quality of the data it produces is never directly measured or analyzed. The paper provides no evidence to show that the generated demonstrations are indeed \"expert\" or how they compare to demonstrations collected via other methods, such as human teleoperation. The value is thus only implicitly suggested through the HALO policy trained on its data.\n\n**Note**: I acknowledge that this paper possesses a significant amount of effort. The above weaknesses only apply **if the intention is to present the contribution as a unified whole.** Alternatively, and perhaps more effectively, the paper's narrative could be reframed. The work would present as a more coherent and defensible contribution if HALO were positioned as the primary focus, with the environment and data collection pipeline framed as a novel and sophisticated infrastructure built to enable this core research. This would still be a strong contribution, and I will still maintain positive if the authors prefer reframing.\n\n- [Minor]: Citation for PBD is missing: Müller, Matthias, Bruno Heidelberger, Marcus Hennix, and John Ratcliff. \"Position based dynamics.\" Journal of Visual Communication and Image Representation 18, no. 2 (2007): 109-118.\n\nQUESTIONS:\n- The entire data collection process is bootstrapped from a single, manually defined expert demonstration. How does the system perform if this initial demonstration is varied? For instance, if you provide a slightly different (but still successful) folding sequence, how does that affect the diversity and quality of the 100 generated demonstrations and the performance of the resulting policy?\n- Could the authors comment on the extensibility of DexGarmentLab to facilitate future research? For example, what is the workflow for tuning the simulation parameters if a new garment is introduced? The paper mentions parameters were \"carefully tuned\"; is there a systematic procedure for this, or does each new asset require extensive, ad-hoc manual tuning? How readily can the policy and data collection frameworks be adapted to different hardware? How hard would it be to map the 60-DoF action space and the LeapMotion data collection to a new bimanual setup with different kinematics?\n\nLIMITATIONS:\nYes, the authors have done a thorough discussion of the limitations in Appendix F.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 2, "DATE": "2025-06-19"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper presents DexGarmentLab, a data generation pipeline for garment manipulation and a generalizable manipulation policy using dexterous hands. To eliminate the extensive human effort typically required in imitation and reinforcement learning, the authors propose an automated data collection strategy that enables the generation of large-scale data from a single expert demonstration. An affordance model is employed to transfer manipulation strategies from a source garment to diverse target garments. Leveraging the generated dataset, the authors introduce a generalizable policy framework capable of performing garment manipulation across different scenarios. Extensive experiments are conducted to validate both the effectiveness of the data generation process and the generalizability of the learned policy. Real-world evaluations further demonstrate the practical applicability of the proposed approach.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths\n\n- Good motivations and valuable insights. The authors propose an efficient and automatic garment manipulation data collection strategy where only one expert demonstration is required, based on the observation that leveraging RL or IL to collect data would be quite time-consuming and human-labor intensive. \n- Well-designed method. The data generation pipeline effectively leverages affordance modeling to transfer manipulation strategies from one object to new objects. The policy designed to achieve generalization across different garments is also reasonably designed. \n\nWeakness\n\n- The task difficulty is restricted. Although the task involves long-horizon cloth folding using dexterous hands. The manipulation challenge is not that high --- such manipulation could also be completed by two grippers. The value of dexterous hands, as well as their superiority over parallel grippers, are not clearly demonstrated or justified in the context of the garment manipulation tasks considered in this work. \n- Experiments lack a comprehensive comparison with RL-based approaches. Although RL requires reward tuning, there are many works that have explored how to leverage LLMs and VLMs to accelerate the reward tuning process. Compared to IL with demonstrations generated by the proposed approach or collected via teleoperation, RL policies can automatically explore diverse solutions to complete the task and would be much more robust. It would be better not to exclude such types of baselines. \n- Section 3 takes too much space -- more than 4 pages, which leaves quite insufficient space for the experiment section. The method section and the related work section could be made more concise than its current version.\n\nQUESTIONS:\n- Why could the authors claim their GAM as a \"generalizable policy\"? Using which mechanisim could the authors truly guarantee the generalization ability or \"ensure the generalization for xxx\" (line 217)? Except for experiments conducted on cloth with different materials and deformations, the paper does not provide deeper analysis—either from a theoretical perspective or through more fundamental insights—to support the generalization claim. I understand that the method is developed to improve the generalizability. In this case, it would be better not to use such a strong tone since the method cannot fully gaurantee 100% generalization ability.\n- How to generate plausible hand grasping poses in the automated data collection process? The authors mentioned LeapMotion in 4.2. However, after checking the appendix for details, LeapMotion seems to be a teleoperation interface that transfers human hand poses to the shadow hand poses. If the data collection is a teleoperation, then everything is reasonable. But it is claimed as an \"automated\" one.  So what's the automatic grasping pose generation process using LeapMotion? Besides, how to make sure that the grasping pose is physically valid? Would a filtering mechanism be introduced to ensure the plausibility and task completion? If so, what about the success rate of the demonstration generation?\n- Real-world evaluations are conducted on a Psi Bot with dexterous hands, which are different from the shadow hand. So, how to directly transfer the policy trained on shadow hands to the real robot? Would it require retraining the policy using Psi Bot in the simulator? If so, we need to re-collect demonstrations -- and since the LeapMotion is designed for shadow hand -- how to generate grasping poses for Psi Bot?\n\nLIMITATIONS:\n- Insufficient comparisons. The paper lacks comparisons with reinforcement learning (RL)-based approaches and imitation learning (IL) strategies that use demonstrations collected via teleoperation. Without these baselines, it is difficult to fully assess the effectiveness and advantages of the proposed demonstration generation method and policy training strategy.\n- Presentation quality. (1) Sections 2 and 3 occupy a disproportionate amount of space relative to their content. (2) Some claims throughout the paper are stated too strongly, which may overstate the method's contributions. (3) Several figures could be made more concise to enhance clarity and improve the overall flow.\n- Restricted scientific value. (This is a relatively minor point and has little impact on my overall rating, but I include it for completeness.) The technical innovations and the core message of the paper do not appear to offer fundamentally new insights or contributions to the field.\n\nOverall, DexGarmentLab proposes a promising manipulation environment tailored for a range of tasks involving deformable garments. The thoughtful design of the simulation setup, task environment, garment assets, and the proposed policy training strategy collectively contributes a useful resource to the community. So I would like to give a positive rating. Considering limitations like insufficient experimental comparisons, issues with presentation quality, and a relatively modest level of scientific value, my support would be limited to BA.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 2, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-27"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper is dedicated to solving the dexterous garment manipulation problem. It introduces a new simulation environment (DexGarmentLab), an effective method for automatically collecting demonstration data from a single expert example, and a hierarchical policy (HALO) that generalizes across diverse garments. Extensive experiments in both simulation and the real world validate the approach, showing it significantly outperforms existing methods.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths**\n\n- The paper delivers a complete framework—a novel simulated environment, an automated data pipeline, and a new dexterous garment manipulation policy—creating a valuable resource that can accelerate research in this field.\n- The proposed DexGarmentLab is a novel and large-scale simulation environment, which features over 2,500 garments, 15 task scenarios.\n- The proposed automated data collection method uses garment structure to generate diverse demonstrations from a single example, which effectively solves a major bottleneck for imitation learning.\n- The method's effectiveness is convincingly demonstrated through rigorous experiments on 14 simulated and 4 real-world tasks, with clear ablation studies and significant improvements over some baselines. \n\n\n**Weaknesses**\n\n- The methodological comparison is limited to Diffusion Policy (DP) and 3D Diffusion Policy (DP3). Including other relevant imitation learning or deformable object manipulation methods would provide a more comprehensive performance benchmark.\n- The quality of the automatically generated data is not quantitatively compared against traditional data collection methods (e.g., human teleoperation), which could further validate the pipeline's effectiveness. Moreover, the paper emphasizes the efficiency of the automated data collection pipeline. What is the success rate of the pipeline in generating valid and successful task demonstrations that can be used for policy training?\n- A more in-depth analysis of the sim-to-real transfer, including specific challenges and the impact of perception noise in the real world, would provide greater insight.\n- The paper would benefit from a clearer discussion on the boundaries of the policy's generalization, including an analysis of specific failure cases on more complex or unusual garments.\n\nQUESTIONS:\nsee the weaknesses\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-30"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe authors introduce DexGarmentLab, a novel simulation environment designed for dexterous (especially bimanual) garment manipulation. Built on Isaac Sim, it offers over 2,500 garments across 8 categories and 15 task scenarios with high-quality 3D assets. They also propose an automated data collection pipeline that generates diverse demonstrations using a single expert demonstration, reducing manual intervention. Furthermore, the authors present HALO (Hierarchical gArment manipuLation pOlicy), a generalizable policy that combines affordance points for locating manipulation areas and a diffusion method for generating trajectories. Through extensive experiments, the authors demonstrate that HALO outperforms existing methods in both simulation and real-world settings, showing strong generalization across various garment shapes and deformations.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths：\n1. DexGarmentLab Environment: Offers a realistic and diverse simulation platform for garment manipulation tasks.\n2. Automated Data Collection: Reduces manual effort by generating diverse demonstrations from a single expert demonstration.\n3. HALO Policy: Combines affordance points and a diffusion method to achieve strong generalization across different garment shapes and deformations.\n4. Experimental Validation: Demonstrates superior performance and generalization ability of HALO in both simulation and real-world experiments.\n\nWeaknesses：\n1. While the authors compare DP and DP3 methods, the study lacks benchmarking against state-of-the-art VLAs (pi0 or RDT) that demonstrate superior performance on folding tasks.\n2. While demonstrating performance gains on this benchmark DexGarmentLab, the claimed generalizability of the proposed method requires validation across diverse simulation environments. Additional cross-platform testing would strengthen the evidence for its universal applicability.\n3. Could the authors clarify whether this is a single-task or multi-task setting?\n4. Please specify the computational costs (time overhead) for both training and inference phases of the algorithm.\n\nQUESTIONS:\nSee Weaknesses\n\nLIMITATIONS:\nSee Weaknesses", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 2, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces DexGarmentLab, a novel simulation environment specifically designed for dexterous (especially bimanual) garment manipulation. The environment provides large-scale high-quality 3D 8 assets for 15 task scenarios, and refines simulation techniques tailored for garment 9 modeling to reduce the sim-to-real gap. To address the bottleneck of labor-intensive data collection, the authors propose an automated pipeline that leverages a single expert demonstration and a GAM to generate diverse manipulation trajectories for novel garments. Furthermore, they propose HALO, which combines affordance point inference and a structure-aware diffusion policy to Generate Generalizable Trajectories. Extensive experiments show that the methods outperforms baselines, and generalizing well to unseen garments,shape,deformation,env config.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths\n\n1.\tThe integration of GAM and structure-aware diffusion policy (HALO) is novel and provides a principled way to generalize manipulation across unseen garments and shapes.\n2.\tThe proposed environment and methods are highly relevant for real-world applications in robotics, such as laundry folding, dressing, and assistive robotics.\n3.  The figures and tables are of high quality, with clear and informative visualizations that greatly aid understanding.\n\nWeaknesses\n\n1.\tIn the real-world experiments, particularly on the Fold Tops and Hang Tops tasks, the garments used appear to have relatively simple and uniform colors. It is unclear how well the proposed method would generalize to garments with more complex patterns, prints, or highly varied textures, which are common in real-world scenarios.\n2.\tIn Table 3, please clarify what metric is being reported (e.g., is it the number of successful trials out of total attempts, or another measure?).\n\nQUESTIONS:\n1.\tYour method uses five-finger dexterous hands instead of common two-finger grippers. What specific advantages do five fingers provide for garment manipulation tasks in your experiments?\n2.\tAre the grasp points on the garment defined in 2D (on the surface) or 3D coordinates?\n3.\tSince garments are flexible, did you consider grasping force or pressure to avoid slippage or fabric damage?\n\nLIMITATIONS:\nThe authors have adequately addressed the limitations.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 4, "IMPACT": 4, "ORIGINALITY": 3, "DATE": "2025-07-03"}