Title: DexFlyWheel: A Scalable and Self-improving Data Generation Framework for Dexterous Manipulation
Venue: NeurIPS 2025
Accepted: yes
Keywords: robotic learning

dexterous manipulation

data flywheel
Abstract: Dexterous manipulation is critical for advancing robot capabilities in real-world applications, yet diverse and high-quality datasets remain scarce. Existing data collection methods either rely on human teleoperation or require significant human engineering, or generate data with limited diversity, which restricts their scalability and generalization. In this paper, we introduce DexFlyWheel, a scalable data generation framework that employs a self-improving cycle to continuously enrich data diversity. Starting from efficient seed demonstrations warmup, DexFlyWheel expands the dataset through iterative cycles. Each cycle follows a closed-loop pipeline that integrates Imitation Learning (IL), residual Reinforcement Learning (RL), rollout trajectory collection, and data augmentation. Specifically, IL extracts human-like behaviors from demonstrations, and residual RL enhances policy generalization. The learned policy is then used to generate trajectories in simulation, which are further augmented across diverse environments and spatial configurations before being fed back into the next cycle. Over successive iterations, a self-improving data flywheel effect emerges, producing datasets that cover diverse scenarios and thereby scaling policy performance. Experimental results demonstrate that DexFlyWheel generates over 2,000 diverse demonstrations across four challenging tasks. Policies trained on our dataset achieve an average success rate of 81.9\% on the challenge test sets and successfully transfer to the real world through digital twin, achieving a 78.3\% success rate on dual-arm lift tasks.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes a data flywheel approach to training robots in simulation. A VR teleoperation interface collects a small set of human demonstrations that are used to train a behavior cloning policy. This base policy is then modified with a residual policy using reinforcement learning in a simulated setup. Data collected with this modified policy is filtered, augmented, and used to improve the base policy using imitation learning. Results show iterative improvement as cycles of the flywheel are run, including transfer to real robot setups.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths \n\n- VR teleoperation with bimanual hand tracking makes human data collection relatively straightforward\n- meaningful results with sim2real transfer\n- results show improvement in generalization tests with each iteration of the flywheel\n\nWeaknesses \n\n- although this approach requires comparatively few human demonstrations, it also requires access to ground truth simulation reward and carefully engineered reward functions\n- Requires simulation and carefully engineered rewards to work with the RL part\n- The strong claim of “infinite diverse data generation only from one demonstration per task” does not match the experiments, which collect 20 trajectories per task in the first iteration. It is an overclaim.\n- Performance drop in T_O^i (Table 2) with added data does not make sense and not much explanation is given about this observation. It is especially surprising given that the tasks require dexterous manipulation.\n- filtering requires success detection, which is accessible in simulation but not in the real world without additional modues\n- There exist a growing body of work focusing on data flywheel-type approaches, but there are limited mentions of these works. These works below may not all be fully relevant, but provided here for convenience\n    - https://arxiv.org/pdf/2411.01813v1\n    - https://arxiv.org/abs/2405.10315\n    - https://arxiv.org/pdf/2407.16677v1\n    - https://arxiv.org/pdf/2505.10442v1\n    - https://proceedings.mlr.press/v164/lu22a/lu22a.pdf\n    - https://arxiv.org/pdf/2410.14868v4\n\nQUESTIONS:\n- If DexFlyWheel requires simulators and rewards, how is this different and better than other RL at scale works like MT-Opt (https://arxiv.org/pdf/2104.08212) that also have a filtered data loop?\n- Why are initial success rates at iter i = 1 (Table 2) higher than the initial SR in iters i = 2 and i = 3? The near 100% success rates feel odd given that the policy has only been trained on 20 trajectories, especially because the robot is using  What can explain the high initial success rates and the initial ability for the robot to learn from 20 trajectories only?\n- What exactly is your data augmentation module A_EP doing? It is briefly mentioned in lines 149-155 but it seems like a contribution in the pipeline, so it is worth discussing in more detail if possible.\n\nI’d be willing to increase the scores and overall rating if the authors present a compelling argument on the originality of this approach and its advantages over other continual RL/IL paradigms, as well as clarified why TO^i results start high and decrease.\n\nLIMITATIONS:\n- Specify the need for simulations for this pipeline to work (given the amount of residual RL and rewards needed)\n- Reduce the strength of the paper claim: “infinite diverse data generation only from one demonstration per task”", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 2, "DATE": "2025-06-25"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper introduces DexFlyWheel, a scalable framework for generating high-quality and diverse dexterous manipulation datasets from minimal human demonstrations. It combines imitation learning (IL) and residual reinforcement learning (RL) in a self-improving cycle. Starting from a single human demonstration per task, DexFlyWheel iteratively augments data diversity and policy generalization by rolling out trajectories in simulation, applying domain randomization, and refining policies via RL. The method is validated on multiple dexterous manipulation tasks (grasp, pour, lift, handover) and achieves strong performance both in simulation and real-world deployment\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n- Novelty: Introduces a closed-loop “FlyWheel” mechanism integrating IL and RL, enabling scalable data generation from minimal seed demonstrations.\n- Highly Scalable: Achieves over 2000 diverse demonstrations from just one teleoperated demo per task.\n- Extensive simulation experiments with multiple baselines and ablation studies.\n- Demonstrates promising sim-to-real transfer on dual-arm Lift tasks.\n\nWeakness:\n- While the simulation comparisons are thorough, it would significantly strengthen the claims of superior real-world generalization if baseline methods used in simulation were also deployed on hardware. This would help quantify the sim-to-real gap across methods and ensure fair evaluation.\n- While the real-world deployment of DexFlyWheel on dual-arm tasks demonstrates promising sim-to-real transfer, no real hardware results are reported for single-arm tasks (Grasp and Pour). This omission limits the ability to fully assess the framework’s generalization performance across all studied tasks.\n- The framework’s evaluation covers 4 dexterous manipulation tasks, which provides initial evidence of scalability. However, these tasks are limited. It would be helpful to add more tasks.\n\nQUESTIONS:\n- Why were only the dual-arm tasks (Lift, Handover) deployed in real-world experiments? Were there specific challenges in deploying single-arm tasks (Grasp, Pour)? Including even preliminary hardware results for single-arm tasks would strengthen claims of broad applicability.\n- The absence of real-world deployment for baseline methods limits comparative insights into sim-to-real performance. Could the authors discuss any technical or fairness challenges that prevented such evaluation? Would they consider adding such comparisons in future work or supplementary material?\n- The current evaluation uses four rigid-object tabletop tasks. Can the authors comment on the feasibility of extending DexFlyWheel to tasks with articulated objects, or long-horizon manipulation? This would provide a more robust test of the framework’s scalability.\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 4, "ORIGINALITY": 3, "DATE": "2025-07-03"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper addresses the problem of how to generate data at-scale with limited human teleoperation time. The key insight of this paper is that manipulating different objects induces minor changes in trajectories. With this, the authors present DexFlyWheel, which is a continual learning and data collection pipeline to generate diverse and high-quality dexterous manipulation data from few actual teleoperation data. To evaluate DexFlyWheel, the authors compared it against DexMimicGen, another synthetic data generation method, and showed extensive improvement for four tasks.\n\nSTRENGTHS AND WEAKNESSES:\nPositives\n* Very impressive task results across multiple tasks, compared to baselines (DexMimicGen)\n* Experimental setup is sound, covering expected baselines\n* Approach is sound and intuitive (leveraging a continual learning framework while learning a residual policy to handle the slight differences seems to be a good way to decompose the problem)\n\nAreas of improvement\n* Comment and experiments showing if this is limited to manipulating things that can be well modeled by simulator? The approach relies heavily on a simulator's ability to accurately capture the motions of the object upon interaction. The current experiments are on relatively easy-to-model objects. \n* Comment and experiments on where this insight of \"manipulating different objects only induces minor changes in the manipulation\" break, for example, lifting geometrically similar but much heavier object.\n* Typo: \"desgin\", numbering in baselines, there are two (4)s, \".(detailed analysis in Section 5.4.)\"\n\nQUESTIONS:\n*  How much of this method is on manipulating objects that can be well modeled by simulator? The approach relies heavily on a simulator's ability to accurately capture the motions of the object upon interaction. The current experiments are on relatively easy-to-model objects. I may decrease the score if there is no or an uncompelling written justification. I may raise my score for a good written justification, I will raise my score for a good experimental results.\n\n* How widely applicable is this insight of \"manipulating different objects only induces minor changes in the manipulation\" break, for example, lifting geometrically similar but much heavier object? How does the approach respond to this? The current experiments are on relatively easy-to-model objects. I may decrease the score if there is no or an uncompelling written justification. I may raise my score for a good written justification, I will raise my score for a good experimental results.\n\n* Please fix typo: \"desgin\", numbering in baselines, there are two (4)s, \".(detailed analysis in Section 5.4.)\"\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 4, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-03"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper presents DexFlyWheel, a scalable framework for generating diverse and high-quality dexterous manipulation data in simulation. It starts from a single human demonstration, and then iteratively expands data diversity using a closed-loop pipeline that combines imitation learning, residual reinforcement learning, trajectory rollouts, and data augmentation. The key insight is to use imitation learning to preserve human-like behaviors and using RL and domain randomization to improve generalization. The system generates over 2,000 high-quality demonstrations per task and significantly improves policy success rates in both simulation and real-world evaluations.\n\nSTRENGTHS AND WEAKNESSES:\n(+) The paper presents a compelling and modular pipeline that effectively combines imitation learning, residual RL, and augmentation. The architecture (Figure 2) is clear and reproducible.\n\n(+) The system achieves up to 81.9% success rate on challenging generalization test sets (including novel objects, environments, and spatial setups), and outperforms strong baselines (e.g., DexMimicGen) by 28.7% on average.\n\n(+) The authors validate the pipeline with real-world bimanual robot experiments, achieving high success rates on lift and handover tasks, which is a strong testament to the practicality of the proposed approach.\n\n(+) The contributions of the residual RL module and augmentation module are carefully disentangled through well-structured ablation studies (Table 1, Figure 5), and failure cases are systematically explored (Table 3).\n\n(-) The framework still requires manually defined task-specific reward functions for residual RL. This contradicts the goal of scaling with minimal human input and could limit generalization to new task families.\n\n(-) The system only relies on vision, object state, and proprioception. While well-justified for scalability, this might limit performance on fine-grained, contact-rich tasks that require force or tactile cues.\n\n(-) While Table 3 quantifies collection success rates, the paper lacks qualitative analysis or visualizations of typical failure modes during data generation or real-world execution.\n\n(-) It would be interesting to assess whether policies trained in one setting (e.g., robot hand or camera view) generalize to another, given the framework's goal of large-scale data generation.\n\nQUESTIONS:\nHow does the performance of DexFlyWheel scale with the number of iterations? Is there a diminishing return beyond iteration 3?\n\nWhat was the wall-clock time and compute cost required to complete the three-iteration flywheel for one task?\n\nIs there a reason the real-world deployment only includes bimanual tasks? Would the single-arm grasp/pour tasks work as well?\n\nLIMITATIONS:\nAs acknowledged by the authors, the main limitations are (1) reliance on hand-crafted reward functions for residual RL, and (2) lack of tactile sensing for contact-rich tasks. Additionally, DexFlyWheel has not yet demonstrated generalization across robot morphologies or sensor setups.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-04"}