DexFlyWheel: A Scalable and Self-improving Data Generation Framework for Dexterous Manipulation
Authors: Kefei Zhu
Fengshuo Bai
YuanHao Xiang
Yishuai Cai
Xinglin Chen
Ruochong Li
Xingtao Wang
Hao Dong
Yaodong Yang
Xiaopeng Fan
Yuanpei Chen
Venue: NeurIPS 2025 spotlight ()
Keywords: robotic learning
dexterous manipulation
data flywheel
Primary area: reinforcement_learning
TL;DR: An efficient data generation framework for dexterous manipulation.

Abstract: Dexterous manipulation is critical for advancing robot capabilities in real-world applications, yet diverse and high-quality datasets remain scarce. Existing data collection methods either rely on human teleoperation or require significant human engineering, or generate data with limited diversity, which restricts their scalability and generalization. In this paper, we introduce DexFlyWheel, a scalable data generation framework that employs a self-improving cycle to continuously enrich data diversity. Starting from efficient seed demonstrations warmup, DexFlyWheel expands the dataset through iterative cycles. Each cycle follows a closed-loop pipeline that integrates Imitation Learning (IL), residual Reinforcement Learning (RL), rollout trajectory collection, and data augmentation. Specifically, IL extracts human-like behaviors from demonstrations, and residual RL enhances policy generalization. The learned policy is then used to generate trajectories in simulation, which are further augmented across diverse environments and spatial configurations before being fed back into the next cycle. Over successive iterations, a self-improving data flywheel effect emerges, producing datasets that cover diverse scenarios and thereby scaling policy performance. Experimental results demonstrate that DexFlyWheel generates over 2,000 diverse demonstrations across four challenging tasks. Policies trained on our dataset achieve an average success rate of 81.9\% on the challenge test sets and successfully transfer to the real world through digital twin, achieving a 78.3\% success rate on dual-arm lift tasks.

FexFllWWeel A Scalable and Self-improving Data Generation Framework for Dexterous Manipulation
Kefei Zhu 1244 Fengshuo Bai*, YuanHao Xiang', Yishuai Cai*, Xinglin Chen', Ruochong Lit,
Xingtao Wang
Hao Dong', Yaodong Yang Xiaopeng Fan 1,st Yuanpei Chen
Harbin Institute of Technology 'PsiBot 'Peking University
PKU-Psihot Lab 'Harbin Institute of Technology Suzhou Research Institute
kefei. meeei.zhu2210g0m www.221mmmml.ccommmmmmm
gaadong-yanggkeee edu. en fxpehit edu. piampel.chen3122gmai... com
Project Page: htts://DexFlyyheeeggiubb..
Teleoperation System
Data Generation Simulation
DexFlyWhee!
Real World Deployments
Diverse Simulation Dataset
Extensive and Diverse Data Generation from Single Demonstration per Task
Figure 1:
Scaling dexterous manipulation data - DexFllyheel generates diverse, high-quality dexterous manipulation data for challenging tasks. Our generated dataset enables policies generalize to unseen scenarios and successfully transfer to the real world.
Abstract
Dexterous manipulation is critical for advancing robot capabilities in real-world applications, yet diverse and high-quality datasets remain scarce. Existing data collection methods either rely on human telcoperation or require significant human engineering, or generate data with limited diversity, which restricts their scalability and generalization. In this paper, we introduce DexFlyWheel, scalable data generation framework that employs self-improving cycle to continuously enrich data diversity. Starting from efficient seed demonstrations warmup, DexFlyWheel expands the dataset through iterative cycles. Each cycle follows a closed-loop pipeline that integrates Imitation Learning (IL), residual Reinforcement Learn- ing (RL), rollout trajectory collection, and data augmentation. Specifically, IL extracts human-likk like behaviors from demonstrations, and residual RL enhances
Corresponding author
'This work was completed during an internship PsiBot.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

policy generalization. The learned policy is then used to generate trajectories in simulation, which are further augmented across diverse environments and spatial configurations before being fed back into the next cycle. Over successive itera- tions, self-improving data flywheel effect emerges, producing datasets that cover diverse scenarios and thereby scaling policy performance. Experimental results demonstrate that DexFlyWheel generates over 2,000 diverse demonstrations across four challenging tasks. Policies trained on our dataset achieve an average success rate of 81 119% on the challenge test sets and successfully transfer to the real world through digital twin, achieving 78.3% success rate dual-arm lift tasks.
Introduction
Learning from Demonstration (LfD) [1] has become increasingly prevalent in robotics. Recent works has shown that training large models with extensive datasets can achieve more challenging tasks and better generalization [2-8]. In dexterous manipulation particularly [9-12], the higher degrees of freedom and richer contact interactions demand larger, more diverse and higher-quality datasets. However, collecting such datasets remains considerable bottleneck. Human teleoperation approaches require significant human effort and typically constrain data collection to laboratory settings, which limits the scalability of data collection. While portable motion capture devices [13] can collect data in- theldd, they still require substantial human involvement and suffer from cross-embodiment gap. Recently, simulation has emerged as a promising solution to address data challenges in robotics [14- 19]. It offers numerous advantages: parallel data collection at scale, easy modification of robot embodiments and sensor configurations, and domain randomization for data augmentation. However, existing simulation- based approaches, such as optimization or heuristic planning methods [20], LLM-driven methods [19, 21-23], and purely RL-based methods [18, 24-26), struggle with the high-dimensional complexity of dexterous manipulation and often produce low-quality trajectories.
Given these simulation challenges, researchers have begun exploring the teleoperation with replay- mechanism [14, 27], where humans telcoperate simulated robots to collect training data and then use spatial transformations to synthesize new trajectories. Although this approach captures relatively high-quality data with simulation- based augmentation, it has several fundamental critical limitations: (I) Inability to Explore Novel Manipulation Strategies. By replaying human demonstrations, these methods confine exploration to existing behaviors, restricting data to the scope of the original demonstrations and hindering generalization to novel scenarios. (2) Insufficient Data Diversity. Since these methods primarily apply spatial augmentations, the generated datasets often exhibit insufficient variability in object geometries and environments, inherently constraining the generalization of learned policies. These limitations motivate to rethink the role of human demonstrations in data generation pipelines. We observe that manipulating different objects typically induces only minor changes in the manipulation trajectories. This suggests regarding human demonstrations not merely as replay data, but as strong behavioral priors that can guide exploration in novel scenarios.
Building on this insight, we propose DexelyWheel, a scalable and infroving data generation paradigm for dexterous manipulation. Our framework features two key design: IL + residual RL for Human-like and Diverse Data Generation. DexFly Wheel combines IL to learn human-like like behaviors from demonstrations with residual RL to adapt these priors to novel scenarios, particularly when manipulating different objects, thereby generating diverse and human-like data. A Dexterous Manipulation Data Flywheel. Inspired by iterative self-improvement in LLMSS [28, 29], we design a data flywheel for dexterous manipulation. Specifically, IL and residual RL are combined with policy rollouts and data augmentation to form a self-improving cycle. At each iteration, the policy generates trajectories, which are then augmented in progressively more diverse scenarios and subsequently fed into the next iteration. This cycle produces a flywheel effect, progressively expanding data diversity, enhancing policy generalization, and evolving into robust, generalizable data generation agent.
In summary, our main contributions include:
We propose DexFlyWheel, scalable and self improving data generation framework for dexterous manipulation. By combining IL. with residual RL and leveraging data augmentation within a self-improving flywheel mechanism, our framework efficiently produces diverse, high-quality demonstrations while preserving human-like behavior patterns. This alleviates the scarcity of dexterous manipulation data and provides solid foundation for training generalizable policies.

---

We demonstrate the effectiveness of our framework on four dexterous manipulation tasks. Starting from single human demonstration per task. DexFlyWhese generates over 2,000 successful demonstrations across 500+ diverse scenarios. The fywheel effect of our framework progressively expands data diversity, enabling it to significantly outperform baseline data generation methods.
We validate that policies trained on our generated data achieve an average success rate of 81.9% on challenging test sets, significantly outperforming policies trained with baselines. Furthermore, our policies transfer to real-world dual -arm robot system via digital twin, achieving 78.3% success rate on the dual-arm lift task and a 63.3% success rate on the dual-arm handover task.
2 Related Work
Dexterous Manipulation. Dexterous manipulation with multi-fingered robotic hands remains a significant challenge in robotics [30-35], largely constrained by high-quality demonstration data scarcity. While the prevailing approach employs reinforcement learning to develop manipulation skills, this method frequently encounters efficiency limitations and exploration challenges [36-39]. Researchers have explored human video demonstrations [40-47], but morphological differences between human and robotic hands create substantial transfer barriers. Human teleoperation has emerged as a promising alternative for collecting expert trajectories for imitation learning [14, 27, 48- 51], effectively capturing expert actions in native robot morphology. Nevertheless, existing approaches still struggle with data collection efficiency or require extensive human engineering, emphasizing the need for high-quality dexterous manipulation datasets.
Robotic Data Generation in Simulation. Current approaches for collecting robotic demonstrations in simulation face significant limitations when applied to dexterous manipulation. Motion planning- based methods, while effective for gripper-based systems [15, 20, 24, 52-54J, struggle with the high-dimensional action space and complex contact dynamics of multi-fingered manipulation. LLMs- driven methods [19, 21-23] can generate high-level command, but they demonstrate limitations when confronted with wfw.re--f--freeeeeeeeeeeeeee----
dexterous hands, unable to provide the fine- gra-iia guidance necessary for coordinated finger-level control. Other pipelines are designed specifically for grasping [52]. and thus do not generalize well to more complex dexterous tasks. RL-based methods have also been widely adopted [36, 24-26, 18, 53, 55, 56), yet purely RL-trained policies often exhibit non-human-like behaviors, leading to less robust manipulation and increased sim-to-real transfer challenges. Moreover, RL faces exploration difficulties and relies heavily on reward engineering, which is particularly acute in dexterous manipulation. Replay-based methods [14, 27, 57] attempt to edit existing demonstrations to new scenarios but face fundamental scalability constraints, as they merely implement spatial transformations of recorded trajectories without the ability to explore novel manipulation strategies beyond the original demonstrations. For example, when an object's 's geometry changes significantly e.gg, from a sphere a cuboid cuboiddrrplaa- replay-based methods struggle adapt finger trajectories. These collective limitations underscore the critical need for more efficient and scalable methods generate diverse, high-quality dexterous manipulation data in simulation.
3 Task Formulation
To address the challenge of generating high-quality synthetic data for robotic manipulation tasks, we train policy models for each manipulation task in simulation environments and use these policies to collect demostrations. We formulate each manipulation task as a Markov Decision Process (MDP) M (S. A, π, T, T,R. R. G). where S is the state space, A is the action space, is the agent's 's policy, T( 18, a,) is the transition distribution, R is the reward function, 7 is the discount factor, and p is the initial state distribution. The policy я conditions on the current state 8t, and generates robot action distributions a to maximize the likelihood between the future object states (81+1. 8142,..., 8(4T).
4 Method
In this section, we begin with an overview of the WexFlyWheel architecture (Section 4.1) and then detail the two-stage data generation pipeline (Sections 4.2 and 4.3). Together, these components enable scalable collection of diverse and high-quality dexterous manipulation demonstrations.
3

---

Warm-up Stage
Self-improving Data FlyWheel Stage
Residual Policy Training
Base Policy hase
Residual Policy Iress
Data Collection via Teleoperation
Base Policy Training
Rollout Trajectory Collection
Base Policy Tasse
Combined Policy *
Fenniined base Tiees
Dijject
2117 61
-20t dssnn
Datasel
Dataa À spmmentation
Trajectory Filter
Dataset D
Rollout Datasets
Datasel Generation
Iteration
Data ugmentation Ale
Data Filter
Figure 2: DexFly Wheel Framework Overview. The framework has stages: warm-up stage (left) and a self-improving data flywheel stage (right). In the warm-up stage, seed demonstrations from VR teleoperation are augmented form the initial dataset D1. The data flywheel stage operates as closed-loop cycle with four key components:() base policy Thase training to capture nouman-like behaviors, (2) residual policy Tres training to enhance generalization, (3) combined policy Teombined rollouts to generate new trajectories, and (4) data augmentation to further diversify the dataset. As the flywheel iterates, both data diversity and policy capability continuously improve.
4.1 Overview
DexFly Wheel aims to generate diverse and high-quality data across various scenario configurations, providing broad coverage of objects, environments, and spatial variations, while only starting with minimal human demonstrations. Figure 2 illustrates the overall architecture of our framework.
Warm-up stage. A single human demonstration daced is collected via VR-based teleoperation system. This seed demonstration is then processed using multi-dimensional data augmentation module ApP. Given the human demonstration dyeed, the augmentor generates demonstrations with diverse environment and spatial variations to produce the initial dataset Dz.
Self-improving Data Wheel stage. We design data flywheel mechanism to progressively enhance both data diversity and policy performance. This stage comprises multiple iterations i == (1,2.... n 1). at each iteration i, the following steps are executed: (1) An imitation learning policy These is trained on dataset D; (with D, used when = 1). (2) To improve generalization to novel objects, residual reinforcement learning policy T is trained on top of the frozen Thsse yielding a combined policy Fgombiind = These + + (3) The combined policy is deployed in simulation to generate demonstrations under various object configurations, forming a high-quality rollout dataset D. (4) Finally, D further augmented by Agr with environment and spatial variations to produce the dataset D.+1 used in the next iteration.
4.2 Warm-up up Stage
The first stage aims to generate initial dataset D2 via data augmentation module Agr. starting from a single human demonstration d This warm-up stage including two operations:
Data Collection via Telcoperation. To bootstrap the framework with high-quality seed data, design VR-based teleoperation system implemented in simulation using Apple Vision Pro [50] to accurately track human hand, wrist, and head poses. Since large-scale data collection requires heavily

---

human effort, we only need a single demonstration, denoted as doed- This demonstration serves as the sole seed for subsequent data generation. This makes our method highly efficient in terms of human resources while maintaining the quality and diversity of the generated data.
Data Augmentation. To scale and diversify our dataset, introduce data augmentation module AEP, which builds upon the MimicGen framework [27] and extends it to support multi lintiioaimenionall data augmentation across various environments and spatial configurations. It can efficiently augment source dataset 'D through trajectory editing and simulation domain randomization. App takes D and augmented scenario configurations Caug as input, and outputs the augmented dataset D'. In the warmup phase, this process is applied to the seed demonstration: D = (dsdd C1). Through this warmup phase, we establish a foundation of diverse manipulation datasets that serves as the starting point for our iterative data flywheel mechanism.
4.3 Seff-improving Data Fly Wheel Stage
The second stage implements a closed-loop data flywheel mechanism that iteratively expands data diversity across objects, environments and spatial generalization dimensions. At each iteration i E (1.2...., n 13, the data flywheel performs four key operations:
Base Policy Training. Given the dataset D; from the previous iteration, we employ diffusion- based policy [58] as the base policy Thase to learning dexterous manipulation skill, obtaining a strong base policy for subsequent modules. At each step i, the base policy Towe takes the state .s!"." as inputs, where s represents visual input from camera, St contains object
8, {sy .tt shhi state information including 6D pose (position and orientation) and velocities, and sp includes robot proprioception data consisting of joint positions, velocities, and end-effector poses. The policy outputs a sequence of robots actions (a,, at+1...., , a++HH,, H), where H represents the prediction horizon, and each action at consists of the end-effector 6D pose and target joint angles. Implementation details of the policy parameters are provided in Appendix A.
Residual Policy Training. Generalizing to novel objects remains key challenge in imitation learning for robotic manipulation, which often suffers from limited data. We observe that manipulating different objects induces only small changes in the manipulation trajectories', suggesting that well-initialized policy can only require fine-grained adjustments to adapt to new objects. Based on this observation, we propose a residual reinforcement learning framework that builds upon the base policy. Specifically, we train residual policy T|e that takes object state s!th and robot proprioception 5."TP as inputs and generates correction actions Za = (Dar.... A0, H). These corrections, scaled by a, are added to the base policy actions to form the combined policy क commiined = Thse + a Tles where ã, ar + o La, at each timestep. This approach allows the residual policy start from reasonable robot actions from Thase and focus on learning the fine- ome-grained refinements to generalize objects. Implementation details are in Appendix A.2.
To stabilize exploration, we employ the progressive schedule from [59], defining the combined policy during training as:
i Toombined (s) Thase (8) + a (s) with probability e
(I)
Tasse (s)
with probability 1 6
where c serves as a mixing coefficient that linearly increases from 0 to 1 over T steps, gradually shifting control from the base to the residual policy.
Rollout Trajectory Collection. In this module, we employ the frozen combined policy of combinned =
hase + Q- -Tee to perform rollouts in simulation under randomized object configurations: Dÿ {dj {(t,, Y-1 |d ~ T Tcoobiined y=1: 1 where we collect K high-quality trajectories by filtering based on task success. This rollout strategy achieves robust object generalization, while geawetr trajectory editing methods fail to adapt [14].
Data Augmentation. In this module, we employ the previously introduced data augmentation module Agp to efficiently augment data various environment and spatial configurations. Taking the dataset Dó and augmented scenario configurations aug as input, the module produces an expanded dataset: DE+1 App(Di ) The expanded dataset Di+1 is used to train an improved base policy, which serves as the foundation for the next iteration.
See Appendix A.7 for details.
5

---

(a) simulation
(b) objects
(c) spatial
X
x
chie (di-thsironmee onmentt
real world
Figure 3: Experiment Setup. Taking the dual-arm robot system as an example, (a) Our simulation environment. (b) Object diversity expansion across iterations, progressing from single object (i=1) to geometrically similar objects (i=2) and diverse geometries and physical properties objects (i=3). (c)
Spatial diversity, showing the spatial arrangements. (d) Environment diversity, including variations in lighting conditions and tabletop appearances. (e) weal-world environment.
5 Experiments
The experiments are designed to answer the following research questions:
Q1: Data Flywheel Effect. How does DexFlyWheel exhibit self- improving data flywheel in dexter- ous manipulation, continuously enhancing data diversity and policy generalization? (Section 5.2)
Q2: Policy Performance and Data Generation Efficiency. How does DexFlyWheel compare with baselines policy performance, data generation robustness, and time efficiency (Section 5.3)?
Q3: Component Contribution. How does each component of DexFly Wheell contribute quantitatively the overall system performance? (Section 5.4)
Q4: Real-World Deployment. How does DexFlyWhees enable the real-world deployment of bimanual dexterous robot systems? (Section 5.5)
5.1 Experimental Setup
Tasks and Robots. We evaluate our framework across four dexterous manipulation tasks on both single-arm and dual-arm robot settings: (1) Grasp (single-arm): The robot must grasp the target object and lift it to a height greater than 0.2 meters from the tabletop.
(2) Pour (single-arm): The robot manipulates a source container to transfer its contents into target container, requiring controlled pouring of the contained objects from one receptacle to another. (3) Lift (dual-arm): The robot performs collaborative manipulation using its two arms synchronously lift an object to minimum height of 15 cm. (4) Handover (dual-arm): The robot performs an intra-agent handover by transferring an object from one hand to the other through a stable, coordinated motion.
For the single-arm Grasp and Pour tasks, we use a Franka Emika Panda robot arm equipped with an Inspire robotic hand. For the dual-arm Lift and Handover tasks, arm paired with a 6-DoF PsiBot GO-R hand. Dual-arm robot is equipped with a RealSense D435 camera mounted on its head, which provides first-person perspective.
a 7-DoF Real-Man RM75-6F
Data Collection and Environment. For each manipulation task, collected only single demon- stration trajectory using VR teleoperation as our minimal seed data. To ensure the generation of high-quality data, employed OmniGibson [60] rendering to generates high-quality data. We prepared 80 distinct objects across various categories and 12 different environments with varying lighting conditions, tabletop appearances. we set the number of iterations to i = 3. For each task, we generated 20, 100, and 500 trajectories in the three iterations, respectively. Simuation setup is visualized in Figure 3. More detailed data collection and environment setup are provided in Appendix A.3
simulation platform. leveraging its realistic
We evaluate our method using two criteria: (1) data diversity: the number of object variations O environment variations E and spatial variations P in our generated dataset
Evaluation Design.

---

D; in each iteration, and the total number of scenarios (0 x E X P) that our pipeline can cover; (2) generalization performance: the Success Rate (SR) of task execution when policies trained on our generated datasets D;. It is calculated as the ratio of successful task completion to the total attempts. Specifically, we construct two types of test sets. First, the multi-factor generalization test set (TOEP) contains 40 unseen scenario configurations that simultaneously incorporate all three types of variations: object, environment, and spatial arrangements. Second, the object generalization test set (T)(1)) evaluates the success rate of robot manipulating different objects when the scenario is fixed. This test set contains all the objects introduced during the data generation process of the i-th iteration. A higher value of this metric not only indicates better object generalization performance of the policy but also implies a better capability to enhance the diversity of objects in data generation. All success rates reported mean values from 5 independent runs. Detailed compositions of all evaluation sets and success rate calculation method are provided in Appendix A.4.
Baselines. We compared our approach against the following methods: (I) Human Demo (Default); 20 human demonstrations per task in a fixed scenario: (2) Human Demo (Enhanced): 20 demonstra- tions collected across diverse scenarios; (3) DexMimicGen (Default) [14]: A representative method for dexterous data generation that synthesizes trajectories via replay and editing. For fair comparison, we provide it with the same initial dataset as DexFlyWheel-a single demonstration per task. and (4) MexMinieGen (Enhanced): To create a stronger baseline, we provided DexMimicGen with 10 diverse human demonstrations collected from different scenarios. This setup significantly enhances its ability to generalize more scenarios, giving it 10x data advantage over our method. (5) w/o Res: An ablation model featuring the base policy with Agp but excluding the residual policy. (6) w/o ApP: An ablation model maintaining the base policy and residual policy while removing the AEP module. (7) w/o Res. + w/o Age: The minimal baseline configuration consisting solely of the base policy, All methods are evaluated under identical conditions: simulation setups, model architectures, and test sets, ensuring a fair and rigorous comparison.
5.2 Validating the Dexterous Data Flywheel Effect
This section empirically investigates Q1. We demonstrate how DexFlyWheel progressively expands dataset diversity and enhances policies performance trained on the generated data. As shown in the mid columns of Table 1, DexFly Wheel successfully expands data diversity with each iteration. In the final iteration (i=3). our method generates an average of 2,040 various scenario configurations spanning 20 different objects per task all starting from just a single human demonstration per task. Furthermore, the object diversity results (shown in the O column of Table 1) demonstrate our framework's 's capacity for object-level data generation. As shown in the SR of Feombined on TOEP column, we observe that as dataset diversity increases across iterations, the generalization capabilities of trained policies correspondingly improve, achieving an average success rate of 81. 81.9% in iteration a substantial improvement from the initial 16.5% in iteration 1. Additionally, Boost with Tres on To (i) column, our residual policies consistently improve performance on object generalization by 32.1% 1% on average. These results suggest promising potential for DexFlyWheel in enhancing the data diversity and policy performance across different dexterous tasks. More detail with extended iterations can be found in Appendix A.5.
shown in the SR
5.3 Comparison of Policy Performance and Data Generation Efficiency
This section evaluates DexFly DexFllWheel and baselines in both policy performance and data generation efficiency to address Q2.
We use identical diffusion-based policy architectures (Appendix A. and train them on datasets collected from DexFly Wheeel and four baselines introduced in Section 5.1. As shown in Table 2, DexFly Wheel consistently achieves higher success rates than both human teleoperation- based data and replay-based methods such as DexMimicGen. This performance highlights the benefit of our iterative data flywheel mechanism, which progressively expands data diversity with policy improvement. Compared to the Human Demo baseline, which uses 20 teleoperated trajectories per task, DexFly Wheel achieves vastly superior performance 100-8116 81.9% vs. 13.4% average success-wi requiring only a single human demonstration per task, significantly reducing the human effort.
Policy Performance.
Data Generation Success Rate and Time Efficiency. As shown in Table 3, DexFly Wheell achieves high success across all tasks (avg. 89.8%). In contrast, DexMiniGGen performs worse on dynamic
7

---

Table 1: Self-improving Data Generation Process. Left) Evaluated tasks and iteration settings. Middle) Dataset diversity statistics, Right) Success Rate (SR) of policies trained on our datasets.
Settings
Data Diversity
Policy Performance
Task
Iter.
0f
E t
Configs
Traj-
SR Boost Tre. To( 1
SR Teombined Tour
==
20
15.0%
Grasp
==2
2
11
8
10
880
100
71.0%
+431
88.00 +335%
58.0% 4388
12
3960
500
35.0%
89.1%
90.0%
i-
7
20
Pour
9
288
100
58.3%
1415
75.0% 1A1S
55.6%% 1.1%%
12
12
1440
500
58.0%
80.7%
85.8%
=
1
20
13.9%
Lift
i= 2
5
100
50.0%
1.795
83.3%
1:33
44.4% 1: 4.44%
i
26
12
1560
500
68.8%
98.0%
79.4%
i=
1
20
0.8% L.K
Handover
5
1.585
85.7%
17.5%
=
20
12
1200
500
32.1% 1%
+355%
62.5 %
+4.3%
72.5% 4.11%
Avg.
1.0
3.0
2.5
9.5
20
16.5%
Avg.
52.0%
Avg.
20.0
12.0
8.8
2040.0
500
48.5%
82.6%
81.9%
Improvement (1 3)
20.0
4.0%
3,5%
214.7%
25.0%
+396.4%
Notes: O: Number of objects, E: Number of environmentts, P: Number of poses. Configs: total scenario configurations (O x x P). Traj: generated trajectories. Bold denotes the final generated datasets at the last iteration (i 3) and best SR performance achieved. Blue indicates improements from residual policy wies (i.e, Thase - Feombined) on object test set (S))). Pink indicates generalization of Teomlined to unseen object- eeret-ennrooomnt-puuee combinations in TogP.
Table 2: Comparison with Baselines. Success rates of policies trained datasets generated by different methods when tested on multi-factor generalization test set (TOEP).
Method
Grasp
Pour
Lift
Handover
Avg.
Human Demo (Default)
6.1 1.2
16.7%
13.9%#219
1.8%
9.4% 13.4%
Human Demo (Enhanced)
15.0%
36. t3.3%
2.5%
0%
DexMimieGen (Default)
30.3% +.8s
38.9%
28.2 4..5%%
28.3%
31.4%
DexMimieGen (Enhanced)
50.3%
44, 13.4%%
43.7% 2..6%
42.5% 22.5%%44499
45.2%
Ours
90.0% % +-3.2%
85.8 % +..5%
79.4% 4 +.7.%%
72.5 % +-4.1%
81.9 %
tasks like Handover (14.8%). DexFlyWhee ensures robust data generation due to its policy-in -he-- loop design and continuous ieerovmppoveett.. For data collection time, we evaluate data collection time for each method under the most challenging setting at the final iteration As shown in Table 4, DexFlyWheel requires only 15s per trajectory. Collecting 500 successful trajectories takes 2.4 hours-ll... 1.83x faster than DexMimicGen and over 5.21x faster than human teleoperation. See Appendix A.6 for more time details.
3) on Lift task.
5.4 Ablation Study on DexFily Wheel Components
This section empirically investigates Q3. We conduct an ablation study across four manipulation tasks to isolate the impact of key modules. As shown in Figure 4, removing the residual policy leads to the most significant drop in task success rates, confirming its critical role in improving generalization and robustness. To further analysis DexellWeell's generalization ability, we compare the number of distinct objects successfully manipulated during data generation (Figure 5). This figure demonstrates that DexFly Wheel achieves superior object diversity compared to DexMimicGen. This performance can be attributed primarily to the residual reinforcement learning module, evidenced by the significant drop in performance when this component is removed (w/o Res vs. ours: from 8.25 to 20 objects average). In contrast, DexMinieGen typically operates effectively only on geometrically similar objects (same categories and shapes), which limits their generalization due to its lack of adaptability and inability to explore new strategies.
8

---

Table 3: Data Generation Success Rate. Success rates of generating successful demonstrations using different methods across tasks.
Method
Grasp
Pour
Lift 90.2%
Handover
Avg.
DexMimicGen 87.3%
81.5%
68.2%
14.8%
63.0% 89.8 %
DexFlyWheel (Ours) 93.6%
89.5%
85.7%
Table 4: Data Generation Time. Comparison of per-trajectory generation time and the total time
required to collect 500 successful trajectories (single RTX 4090 GPU).
Method
Time per Trajectory
Time for 500 Successful Trajectories
Human Teleoperation
60s 15s
12.5
DexMimicGen
4.4 h
DexFly Wheel (Ours)
15s
2.4 h
100
Ape
vie_Ar Ap
25
Texe Dhew Datiariii
Dodtyyttc
80
(5)
Number Object
60
12
12
2254
12
Grasp
Pour
Lift
Handover
Grasp
Pour
Lift
Handover
Figure 4: Ablation Study. Quantitative contri- bution of each module in DexFly Wheel across four manipulation tasks.
Figure 5: Comparison of Object Diversity. Our method successfully handles objects with diverse geometries, sizes, and categories.
5.5 Deployment on Dual-arm Real Robot System
To address Q4, we transfer our trained policy simulation into real-world scenarios through digital twin. We set up identical hardware settings both in simulation and real-world as shown in Figure 3, which includes two Realman RM75-6F arms paired with two PsiBot GO-R hands. We employ egocentric-view RealSense D455 camera to obtain the real-world object pose leveraging FoundationPose [61]. In particular, we train the policy using the dataset generated by the final iteration of DexFlyWhhel and deploy in the real world through the digital twin, ensuring consistency between simulation and physical environments. We evaluate the performance our pipeline on the Dual-arm Lift and Handover Task. Experimental results show success rates of 78.3% (Dual-arm Lift) and 63.3% (Handover) in real-world deployment (20 trajectories per trial, 3 trials).
6 Limitations and Future Work
There several limitations on manually designed reward functions. Future research could investigate how LLM- reward generation methods can be efficiently integrated into DexFly Wheel. Second, our policies and simulations currently lack tactile feedback due to the immaturity of tactile sensing and simulation technologies. We plan explore the potential of sensor-based tactile signals for contact-rich tasks.
work. First, the reinforcement learning process currently relies
Conclusion
We present DexFly Wheel, a scalable and self-improving framework for generating diverse, high- quality dexterous manipulation data from minimal seed demonstrations. Our two-stage pipeline first leverages imitation learning to provide behavioral priors, then applies residual reinforcement learning enhance generalization and robustness. This approach progressively expands the data distribution across diverse objects, environments, and spatial layouts. Experiments demonstrate that DexFil Wheel can generate up to 500x more trajectories and 2 4x more distinct scenarios per task. Policies trained on this data achieve an 81.9% success rate in challenging settings, outperforming baselines and successfully transferring real-world robots.

---

Acknowledgements
We thank the reviewers for their constructive feedback and our labmates for insightful discussions.
References
[1] Stefan Schaal. Learning from demonstration. Advances in neural information processing systems, 9, 1996.
[2] Kevin Black, Noah Brown, Danny Driess, Adnan Esmail, Michael Equi, Chelsea Finn, Niccolo Fusai, Lachy Groom, Karol Hausman, Brian Ichter, Szymon Jakubczak, Tim Jones, Liyiming Ke, Sergey Levine, Adrian Li-Bell, Mohith Mothukuri, Suraj Nair, Karl Pertsch, Lucy Xiaoyang Shi, James Tanner, Quan Vuong, Anna Walling, Haohuan Wang, and Ury Zhilinsky. TO: A vision- assionlaanuaage-cttinn flow model for general robot control. CoRR, abs/2410.24164, 2024.
[3] Anthony Brohan, Noah Brown, Justice Carbajal, Yevgen Chebotar, Xi Chen, Krzysztof Choro- manski, Tianli Ding, Danny Driess, Avinava Dubey, Chelsea Finn, Pete Florence, Chuyuan Fu, Montse Gonzalez Arenas, Keerthana Gopalakrishnan, Kehang Han, Karol Hausman, Alexander Herzog, Jasmine Hsu, Brian lchter, Alex Irpan, Nikhil Joshi, Ryan Julian, Dmitry Kalash- nikov, Yuheng Kuang, Isabel Leal, Lisa Lee, Tsang-Wei Edward Lee, Sergey Levine, Yao Lu, Henryk Michalewski, Igor Mordatch, Karl Pertsch, Kanishka Rao, Krista Reymann, Michael Ryoo, Grecia Salazar, Pannag Sanketi, Pierre Sermanet, Jaspiar Singh, Anikait Singh, Radu Soricut, Huong Tran, Vincent Vanhoucke, Quan Vuong. Ayzaan Wahid, Stefan Welker, Paul Wohlhart, Jialin Wu, Fei Xia, Ted Xiao, Peng Xu, Sichun Xu, Tianhe Yu, and Brianna Zitkovich. RI-2: Vision--anguaag--ctiinn models transfer web knowledge to robotic control, 2023. URL https://arxiv org/ absa 2307. 15818.
[4] Songming Liu, Lingxuan Wu, Bangguo Li, Hengkai Tan, Huayu Chen, Zhengyi Wang, Ke Xu, Hang Su, and Jun Zhu. Rdt-1 a diffusion foundation model for bimanual manipulation. arXiv preprint arXiv:2410.07864, 2024.
[5] Minjie Zhu, Yichen Zhu, Jinming Li, Junjie Wen, Zhiyuan Xu, Ning Liu, Ran Cheng, Chaomin Shen, Yaxin Peng, Feifei Feng, and Jian Tang. Scaling diffusion policy in transformer to billion parameters for robotic manipulation. arXiv preprint arXiv:2409. 100009..4411 14411, 2024.
[6] BAAI RoboBrain Team, Mingyu Cao, Huajie Tan, Yuheng Ji, Minglan Lin, Zhiyu Li, Zhou Cao, Pengwei Wang, Enshen Zhou, Yi Han, al. Robobrain 2.0 technical report. arXiv preprint arXiv:2507.02029 2025.
[7] Yi Han, Cheng Chi, Enshen Zhou, Shanyu Rong, Jingkum An, Pengwei Wang. Zhongyuan Wang. Lu Sheng. and Shanghang Zhang. Tiger: Tool-integran geometric reasoning in Essionlanguaaee models for robotics. arXiv preprint 110.:::511000188. 2025.
[8] Enshen Zhou, Jingkun An, Cheng Chi, Yi Han, Shanyu Rong. Chi Zhang, Pengwei Wang, Zhongyuan Wang, Tiejun Huang, Lu Sheng, et al. Roborefer: Towards spatial referring with reasoning in vision-language models for roboties. arXiv preprint arXiv:2506.044: 043088 2025.
[9] Yunfei Bai and C. Karen Liu. Dexterous manipulation using both palm and fingers. In International Conference on Robotics and Automation (ICRA), pages 1560-1565, 2014. doi: 10.1109/ICRA.000 69900099.
[10] S. Gruber. Robot hands and the mechanics of manipulation. IEEE Journal on Robotics and Automation, 2(1):59-59, 1986. doi: 10.
[11] Vikash Kumar, Yuval Tassa, Tom Erez, and Emanuel Todorov. Real-time behaviour synthesis for dynamic manipuaaipplaiion. In International Conference on Robotics and Automation (ICRA), pages 6808 6815, 2014. doi: 10.1 (00////AAA.....9977750250022
[12] Zhecheng Yuan, Tianming Wei, Langzhe Gu, Pu Hua, Tianhai Liang, Yuanpei Chen, and Huazhe Xu. Hermes: Human-to-robot embodied learning from multi-source motion data for mobile dexterous manipulation, 2025. URL https / //arxiv....vv////a org/ /250 20085.
[13] Chen Wang, Haochen Shi, Weizhuo Wang, Ruohan Zhang, Li Fei-Fei, and C. Karen Liu. Dexcap: Scalable and portable mocap data collection system for dexterous manipulation, 2024.
URL https:/ //arxivv..ii/////a org/ ab0////3..32../////////////3/ 07788.
10

---

[14] Zhenyu Jiang, Yuqi Xie, Kevin Lin, Zhenjia Xu, Weikang Wan, Ajay Mandlekar, Linxi Fan, and Yuke Zhu. Dexmimicgen: Automated data generation for bimanual dexterous manipulation via imitation learning, 2024. URL https: //txxvva.ivvv./.a org/ abs awwaab//aa/ 2410. .41855.
[15] Stephen James, Zicong Ma, David Rovick Arrojo, and Andrew J. Davison. Rlbench: The robot learning benchmark & learning environment, 2019. URL 12271.
[16] Ajay Mandlekar, Soroush Nasiriany, Bowen Wen, Iretiayo Akinola, Yashraj Narang, Linxi Fan, Yuke Zhu, and Dieter Fox. Mimicgen: data generation system for scalable robot learning using human demonstrations, 2023. URL https://///a wwwp:/iii..oooooooooo htsp:/bb/223111 1399..
[17] Soroush Nasiriany, Abhiram Maddukuri, Lance Zhang, Adeet Parikh, Aaron Lo, Abhishek Joshi, Ajay Mandlekar, and Yuke Zhu. Robocasa: Large-scale simulation of everyday tasks for generalist robots, 2024. URL wttpttttts//ai...... wtw.//////////aaeeeeeeeeee 02523.
[18) Yufei Wang, Zhou Xian, Feng Chen, Tsun-Hsuan Wang, Yian Wang, Katerina Fragkiadaki, Zackory Erickson, David Held, and Chuang Gan. Robogen: Towards unleashing infinite data for automated robot learning via generative simulation, 2024. URL https: arrii org/ abs/2311. 01455.
[19] Yao Mu, Tianxing Chen, Shijia Peng, Zanxin Chen, Zeyu Gao, Yude Zou, Lunkai Lin, Zhiqiang Xie, and Ping Luo. Robotwin: Dual-arm robot benchmark with generative digital twins (early version), 2024. URL https:tt///// //arxiv...vi/////// org/ org/abs// /209. .40.//.299.........
[20] Hao-Shu Fang, Chenxi Wang. Minghao Gou, and Cewu Lu. Graspnet- Iiillion: A large- scale benchmark for general object grasping. In Proceedings of the IEEE/CVT Conference on Computer Vision and Pattern Recognition (CVPR), June 2020.
[21] Huy Ha, Pete Florence, and Shuran Song. Scaling up and distilling down: tonguage-guidee robot skill acquisition. In Conference on Robot Learning, pages 3766-3777. PMLR, 2023.
[22] Wenlong Huang, Chen Wang, Yunzhu Li, Ruohan Zhang, and Li Fei-Fei. Rekep: Spatio- temporal reasoning relational keypoint constraints for robotic manipulation. arXiv preprint arXiv:2409.01652. 2024.
[23] Shyam Sundar Kannan, Vishnunandan L. N. Venkatesh, and Byung- Cheol Min. Smart-llm: Smart multi-agent robot task planning using large language models, 2024. URL https: //arxiv.org. Sttrxiv.org/abb//3002 2309/ 10062.
[24] Jiayuan Gu, Fanbo Xiang, Xuanlin Li, Zhan Ling, Xiqiang Liu, Tongzhou Mu, Yihe Tang, Stone Tao, Xinyue Wei, Yunchao Yao, et al. Maniskil12: A unified benchmark for generalizable manipulation skills. arXiv preprint ar arXiv:2302.0465) 2023.
[25] Yuanpei Chen, Yiran Geng, Fangwei Zhong, Jiaming Ji, Jiechuang Jiang, Zongqing Lu, Hao Dong, and Yaodong Yang. Bi-dexhands: Towards human-level bimanual dexterous manipulation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 46(5):2804-281 2023.
[26] Yuanpei Chen, Chen Wang, Li Fei-Fei, and C Karen Liu. Sequential dexterity: Chaining dexterous policies for long-horizon manipulation. arXiv preprint arXiv:2309.00988 2023.
[27] Ajay Mandlekar, Soroush Nasiriany, Bowen Wen, Iretiayo Akinola, Yashraj Narang, Linxi Fan, Yuke Zhu, and Dieter Fox. Mimicgen: data generation system for scalable robot learning using human demonstrations. arXiv preprint arXiv:2310. 17596, 2023.
[28] Haipeng Luo, Qingfeng Sun, Can Xu, Pu Zhao, Qingwei Lin, Jianguang Lou, Shifeng Chen, Yansong Tang, and Weizhu Chen. Arena learning: Build data flywheel for llms post-training via simulated chatbot arena. arXiv preprint wtw.:::240..1.222. 2024.
[29] Eric Zelikman, Yuhuai Wu, Jesse Mu, and Noah D Goodman. Star: Self-taught reasoner bootstrapping reasoning with reasoning. In Proc. the 36th International Conference on Neural Information Processing Systems, volume 1 1126, 2024.

---

[30] Matthew T Mason and Kenneth Salisbury Jr. Robot hands and the mechanics of manipulation. 1985.
[31] Hongming Zhang, Chenjun Xiao, Han Wang, Jun Jin, bo xu, and Martin Müller. Replay memory as an empirical MDP: Combining conservative estimation with experience replay. In The Eleventh International Conference Learning Representations, 2023. URL https:
/ //openreview net/ S jJ2VSSJUt8.
[32] Antonio Bicchi. Hands for dexterous manipulation and robust grasping: A difficult road toward simplicity. IEEE Transactions robotics and automation, 16(6):652-662, 2002.
[33] Igor Mordatch, Zoran Popović, and Emanuel Todorov. Contact-invariant optimization for hand manipulation. In Proceedings of the ACM SIGGRAPPH/Furographiss symposium on computer animation, pages 137-144, 2012.
[34] Vikash Kumar, Yuval Tassa, Tom Erez, and Emanuel Todorov. Real-time behaviour synthesis for dynamic hand- manimuuaaalaiin. Automation (ICRA), pages 6808-6814 IEEE, 2014.
2014 IEEE International Conference on Robotics and
[35] Yunfei Bai and C Karen Liu. Dexterous manipulation using both palm and fingers. In 2014 IEEE International Conference on Roboties and Automation (ICRA), pages 1560-1565. IEEE, 2014.
[36] Fengshuo Bai, Yu Li, Jie Chu, Tawei Chou, Runchuan Zhu, Ying Wen, Yaodong Yang, and Yuanpei Chen. Retrieval dexterity: Efficient object retrieval in clutters with dexterous hand. arXiv preprint arXiv:2502.1 8423, 2025.
[37] Yuanpei Chen, Tianhao Wu, Shengjie Wang, Xidong Feng, Jiechuang Jiang, Stephen Marcus McAleer, Yiran Geng, Hao Dong, Zongqing Lu, Song-Chur Zhu, and Yaodong Yang. Towards human-level bimanual dexterous manipulation with reinforcement learning, 2022. URL https:
//arxiv. org/ abs /2066.2066222 08686.
[38] Chen Tang, Ben Abbatematteo, Jiaheng Hu, Rohan Chandra, Roberto Martín-Martín, and Peter Stone. Deep reinforcement learning for robotics: survey of real-world successes, 2024. URL https://arxis: org/ abs/ 03539.
[39] Hongming Zhang, Ke Sun, bo xu, Linglong Kong, and Martin Müller. A distance-based anomaly detection framework for deep reinforcement learning. Transactions on Machine Learning Research, 2024. ISSN 2835-8856. URL httpss://ppenrrieee www.:/wwnweeoe/oeeeeeme TNKhDBV6PA.
[40] Shikhar Bahl, Abhinav Gupta, and Deepak Pathak. Human-to-robot imitation in the wild. arXiv preprint arXiv:2207.0944 2022.
[41] Priyanka Mandikal and Kristen Grauman. Dexvip: Learning dexterous grasping with human hand pose priors from video. In Conference on Robot Learning, pages 651-661. PMLR, 2022.
[42] Shikhar Bahl, Russell Mendonca, Lili Chen, Unnat Jain, and Deepak Pathak. Affordances from human videos as a versatile representation for robotics. Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pages 13778-13790, 2023.
[43] Kenneth Shaw, Shikhar Bahl, and Deepak Pathak. Videodex: Learning dexterity from internet videos. In Conference on Robot Learning, pages 654 PMLL 2023.
[44] Sumedh Sontakke, Jesse Zhang, Séb Amoll, Karl Pertsch, Erdem Bıyik, Dorsa Sadigh, Chelsea Finn, and Laurent Itti. Roboclip: One demonstration is enough to learn robot policies. Advances in Neural Information Processing Systems, 36:55681-55693 2023.
[45] Robert McCarthy, Daniel CH Tan, Dominik Schmidt, Fernando Acero, Nathan Herr, Yilun Du, Thomas G Thuruthel, and Zhibin Li. Towards generalist robot learning from internet video: A survey. arXiv preprint arXiv:2404. 20..22444.9664, 2024.
[46] Arpit Bahety, Priyanka Mandikal, Ben Abbatematteo, and Roberto Martín-Martie Screwmimic: Bimanual imitation from human videos with screw space projection. arXiv preprint arXiv:2405.0 2Xiv:2405.03666. 2024.
12

---

[47] Hanzhi Chen, Boyang Sun, Anran Zhang. Mare Pollefeys, and Stefan Leutenegger. Vidbot: Learning generalizable 3d actions from in-the-wild 2d human videos for zero-shot robotic manipulation. arXiv preprint -iXi:25033007333 2025.
[48] Lai Sum Yim, Quang TN Vo, Ching-I Huang, Chi-Ruei Wang, Wren McQueary, Hsueh- Cheng Wang, Haikun Huang, and Lap-Fai Yu. Wfh-vr: Teleoperating robot arm to set dining table across the globe via virtual reality. In 2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 4934. IEEE, 2022.
[49] Yuzhe Qin, Wei Yang. Binghao Huang, Karl Van Wyk, Hao Su, Xiaolong Wang. Yu- Lei Chao, and Dieter Fox. Anyteleop: A general vision-based dexterous robot arm-hand teleoperation system. arviv preprint arXiv:2307.04577 2023.
[50] Xuxin Cheng, Jialong Li, Shiqi Yang, Ge Yang, and Xiaolong Wang. Open-television: Teleop- eration with immersive active visual feedback. arXiv preprint arXiv:2407.01512 2024.
[51] Runyu Ding, Yuzhe Qin, Jiyue Zhu, Chengzhe Jia, Shiqi Yang, Ruihan Yang, Xiaojuan Qi, and Xiaolong Wang. -iino-viioonpoo: Real-time bimanual dexterous teleoperation for imitation learning. arXiv preprint arXiv:2407.03160 2024.
[52] Jialiang Zhang, Haoran Liu, Danshi Li, Xinqiang Yu, Haoran Geng, Yufei Ding, Jiayi Chen, and He Wang. Dexgraspnet 2.0: Learning generative dexterous grasping large-scale synthetic cluttered scenes, 2024. URL https: / aaxiv. org/ //////////// /2410. /21./............./
[53] Pu Hua, Minghuan Liu, Annabella Macaluso, Yunfeng Lin, Weinan Zhang, Huazhe Xu, and Lirui Wang. Gensim2: Scaling robot data generation with multi-modal and reasoning Ilms, 2024. URL /attps://arxii org/ arggabs./ a41/ 03645.
[54] Enshen Zhou, Qi Su, Cheng Chi, Zhizheng Zhang, Zhongyuan Wang, Tiejun Huang, Lu Sheng, and He Wang. Code-as-monitor: Constraint-awarc visual programming for reactive and proac- tive robotic failure detection. arXiv preprint #tXiv:2412.04455 2024.
[55] Yunfei Li, Ying Yuan, Jingzhi Cui, Haoran Huan, Wei Fu, Jiaxuan Gao, Zekai Xu, and Yi Wu. Robot generating data for learning generalizable visual robotic manipulation. In 2024 TEEE/RSI International Conference on Intelligent Robots and Systems (IROS), pages 5813-5820. IEEE, 2024.
[56] Fengshuo Bai, Rui Zhao, Hongming Zhang, Sijia Cui, Ying Wen, Yaodong Yang, Bo Xu, and Lei Han. Efficient preference-basee reinforcement learning via aligned experience estimation. arXiv preprint arXiv:2405.18688. 2024.
[57] Zhengrong Xue, Shuying Deng, Zhenyang Chen, Yixuan Wang. Zhecheng Yuan, and Huazhe Xu. Demogen: Synthetic demonstration generation for data-efficient visuomotor policy learning. arXiv preprint arXiv:2502.16932. 2025.
[58] Cheng Chi, Zhenjia Xu, Siyuan Feng, Eric Cousineau, Yilun Du, Benjamin Burchfiel, Russ Tedrake, and Shuran Song. Diffusion policy: Visuomotor policy learning via action diffusion. The International Journal of Robotics Research, page 02783649224123666, 2023.
[59] Xiu Yuan, Tongzhou Mu, Stone Tao, Yunhao Fang, Mengke Zhang, and Hao Su. Policy decora- tor: Model-agnostic online refinement for large policy model. arXiv preprint 2233:22412.133300 2024.
[60] Chengshu Li, Ruohan Zhang, Josiah Wong, Cem Gokmen, Sanjana Srivastava, Roberto Martín- Martín, Chen Wang, Gabrael Levine, Michael Lingelbach, Jiankai Sun, et al. Behavior- Ik: benchmark for embodied ai with 1,000 everyday activities and realistic simulation. In Conference on Robot Learning, pages 80-93. PMLR, 2023.
[61] Bowen Wen, Wei Yang, Jan Kautz, and Stan Birchfield. Foundationpose: Unified 6d pose estimation and tracking of novel objects. arXiv preprint arXiv:2312.08..2 2023.
[62] Tuomas Haannjja, Aurick Zhou, Pieter Abbeel, and Sergey Levine. Soft actor-critic: Off- policy maximum entropy deep reinforcement learning with a stochastic actor, In International conference on machine learning, pages 1861-1870. Pmlr, 2018
13

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: The claims in the abstract and introduction match the core contributions and are supported by the method and results sections.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification:
The paper includes dedicated discussion of limitations.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
the paper. The authors should use their best
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA]
14

---

Justification:
The paper does not include formal theoretical results or proofs, as it focuses on algorithm design and empirical evaluation.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Answer: [Yes]
The paper provides detailed descriptions of the model architecture, training procedure, dataset, and evaluation metrics, enabling reproduction of the main results without requiring access to the code.
Justification:
Guidelines:
The answer NA means that the paper does not include experiments. If the paper includes experiments, a No answer this question will not perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution a specific model and empirical evaluation, may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how reproduce that algorithm.
(b) If the contribution is primarily
model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (e.g., a large language model), then there should either be way access this model for reproducing the results way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reprodueibility may be tricky in some cases, in which authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers have some path reproducing verifying the results.
5. Open access to data and code
15

---

Question: Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
The code are provided in supplementary materials for reproduction of all key experiments, detailed in the supplemental material.
Justification:
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips.. cc/
for more details.
While encourage the release of code and data, we understand that this might not be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https: / /nips. cc/public//eeeccccccccccccccecct hwwolicgguides////ee /Coo//eoeeemeeeee cestsubmmisiooni swiissosisiyyyo for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing much information possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Does the paper specify all the training and test details (c.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Question:
Answer: [Yes]
Justification: The paper specifies all relevant training details, including hyperparameters, optimizer settings, and training schedules, with additional configurations provided in the appendix.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to a level of detail that is necessary appreciate the results and make sense of them.
The full details can be provided either with the code, appendix, or as supplemental material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [Yes]
Justification: We report the standard deviation over multiple runs reflect variability, and clearly indicate in both the main text and figure captions.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by emoor bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
16

---

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
It should be clear whether the error bar is the standard deviation the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative error rates).
If error bars reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: The paper specifies the of NVIDIA RTX4090 GPUs and 4 CPU cores, along with details on training time and batch sizes for all experiments.
Guidelines:
The answer NA means that the paper does include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental rums as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't fidn't make it into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https http//nnurrrp.ppee wttpppp.ppppppppee cc/public////ccccccccccccccccccc http//ubllc//thccceece Eett//tiiisseeeeeeee csGuidelines"
Answer: [Yes]
Justification: The research adheres to the NeurlPS Code of Ethics, with ethical concerns related to data usage, human subjects, or potential misuse identified.
Guidelines:
The answer NA means that the authors have reviewed the NeurIPS Code of Ethics.
the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make to preserve anonymity (e.g., if there is special consid- eration due to laws or regulations their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: The paper considers positive impacts in assistive and robotics, while also noting possible misuse in surveillance and unintended physical harm from inaccurate predictions.
Guidelines:
17

---

The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No. they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negatíve societal impacts include potential malicious or unintended uses (e.g.. disinformation, generating fake profiles, surveillance), fairness considerations (c.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of MI.).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: The paper does not release models or data with high risk for misuse; thus, no special safeguards are necessary.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [NA]
The paper does not use third-party assets that require attribution or license disclosure.
Justification:
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset used and, possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
18

---

For scraped data from a particular source (e.g., website). the copyright and terms of service of that souree should be provided.
assets are released, the license, copyright information, and terms use in the package should be provided. For popular datasets, paperswi thcode. com datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information is not available online, the authors are encouraged to reach out to the asset's s creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification:
This paper does not release new assets.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/mmodel as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification:
This paper does not involve crowdsourcing.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then much detail as possible should be included in the main paper.
According to the NeurlPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals equivalent for research with human subjects
Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Question:
Answer: [NA]
Justification:
This paper does not involve crowdsoureing.
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
19

---

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if' applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Does the paper describe the usage of LLMs if is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, originality of the research, declaration is not required.
Question:
Answer: [NA]
Justification:
The core method development this research does not involve LLMs.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, or non-standard components.
Please refer to our LLM policy (https: pectpa:/nneten /ttp///nen...ppeeeeee 2205 for what should or should not be described.
20

---

Supplementary Material
Table of Contents
A
Technical Appendices and Supplementary Material
22
Overview
22
A.1 Base Policy Implementation Details
23
A.2 Residual Policy Implement Details
23
Data Collection and Environment Setup
25
A.4 Evaluation Test Set and Success Rate Calculation Method
26
A.5 Performance Scaling and Extended Iteration Analysis
27
A.6 Time Efficiency Comparison
28
A.7 The Minor Adjustment Observation and Curriculum Learning Strategy
28
21

---

A Technical Appendices and Supplementary Material
Overview The Appendix contains the following content:
1. Base Policy Implementation Details (Section A)/): Details the implementation of the base policy, including model inputs and outputs, and training hyperparameters.
Residual Policy Implement Details (Section A.2): Describes residual policy implement details and the reward function design for the residual policy across different tasks.
Data Collection and Environment Setup (Section A.3): Outlines the data generation strategy incorporating environment, object, and spatial variations.
4. Evaluation Test Set and Success Rate Calculation Method (Section A.4): Presents the evaluation test set and the success rate calculation method.
Performance Scaling and Extended Iteration Analysis (Section A.5): Provides additional experiments and analysis to examine how performance scales with the number of flywheel iterations, and investigates whether performance improvements exhibit diminishing returns.
6. Time Efficiency Comparison (Section A.6): Compares the wall-clock time efficiency of DexFly- Wheel against baselines, highlighting both training and data generation overheads.
The Minor Adjustment Observation and Curriculum Learning Strategy (Section A.7): Pro- vides the rationale for why DexFly Wheel can generalize to novel objects, describes the curriculum- based generalization strategy and our key observation, presents experimental validations across object mass and shape variations, and discusses the scope and limitations of this approach.
22

---

A.1 Base Policy Implementation Details
This section details our base policy implementation, including model inputs and outputs, training hyperparameters and computing resources.
Model Inputs and Outputs. The base policy input state is denoted as s = { ,t prop), where:
Visual Input st For single-arm tasks, the input is a front view image T! E R221x221x3 For dual-arm tasks, the input is a top view image I E TR 2222x22xx3
Object State St In most tasks, the object state the state of a single manipulated object. For pour tasks, two objects are involved, and the object state is represented by 26-dimensional vector.
state is represented by 13-dimensional vector representing
prop.singlo-amm E R including joint positions (19 dimensions), joint velocities (19 dimensions), gripper state (12 dimensions), gripper velocity (12 dimensions), end-effector position (3 dimensions), end-effector orientation (4 dimensions). For dual-arm tasks, the proprioception is st sprop.dual-arm positions (36 dimensions, 18 per arm), joint velocities (36 dimensions. 18 per arm), gripper states (11 dimensions per gripper), gripper velocities (11 dimensions per gripper), end-effector positions (3 dimensions per arm), and end-effector orientations (4 dimensions per arm)
Robot Proprioception s. For single-arm tasks, the proprioception is s
R230, including joint
Output. The action sequence is denoted
(dte 0t--1 (t+ where H 8. Each individual
action a includes: An end-effector 6D pose at E RR. Target joint angles of hands af q E R", where n = 10 for dual-arm tasks and n = 7 for single-arm tasks.
Training Hyperparameters. Table 5 summarizes all hyperparameter for the base policy training. Computing Resources. All experiments are conducted on NVIDIA A100 GPUs.
A.2 Residual Policy Implement Details
This section details our residual policy implement details, including policy training and reward design.
Policy Training. We employ the Soft Actor-Critic (SAC) algorithm [62] to train residual policy that enhances a pre-trained diffusion-based manipulation policy. The residual approach enables efficient learning by leveraging an existing base policy while exploring additional action refinements. Detailed hyperparameters are provided in Table The residual actor network implemented as a policy decorator that outputs corrections to the base policy's ' actions, allowing for fine-tuning of manipulation behaviors while maintaining the fundamental skills encoded in the base policy.
To ensure effective learning, we implement progressive exploration strategy that gradually introduces the residual policy's influence over time. For the first 1,500 timesteps, only the base policy's 's actions are executed. Between 1,500 and 10,000 timesteps, the probability of including residual actions increases linearly with the global step count, promoting smooth exploration of the action space. All residual actions are scaled by a factor of 0. to maintain stability while allowing for meaningful corrections to the base policy. The training architecture features dual soft Q-networks with target networks updated at a rate of 7 = 0.01 to provide stable value estimation. The entropy coefficient a automatically tuned to maintain a target entropy based on the action space dimension, balaneing exploration and exploitation. Gradient updates are performed after every 5 environment steps with an updates-to-dae to-dat ratio of 0.2, resulting in total of I gradient update per environment step. Gradients are clipped with a maximum norm of 10 to prevent unstable updates. The critic networks evaluate the combined actions assess the overall quality of the agent's 's behavior, while the actor network operates only on the proprioceptive and object state observations to generate residual corrections. This design allows the residual policy to focus on improving specific aspects of the manipulation task without requiring complete knowledge of the base policy's 's inner workings. The training process continues for 1.5 million timesteps, with model checkpoints saved every 10 episodes to track progress and enable resumption of training if needed.
23

---

Table 5: Hyperparameters for Diffusion Policy Training.
Category
Parameter
Value
Action Steps
General
Observation Steps
1
Embedding Dimension
768
Transformer Layers
7
Network
Attention Heads
8
Attention Dropout
0.1
Model Architecture
vit small 126_s32_ 224
Vision Encoder
Pretrained
True
Frozen
False
Noise Scheduler
DDIMScheduler
Diffusion Model
Train Timesteps
50
Inference Steps
16
Batch Size
256
Training
Epochs
200000
Learning Rate
3.0e-4
Weight Decay
1.0e-6
Optimization
LR Scheduler
cosine
Table 6: Hyperparameters for SAC Residual Policy Training
Category
Parameter
Value
Actor Network (MI.P Layers)
[256, 256, 256] [256, 256, 256]
Critic Network (MLP Layers)
Network Architecture
State Dimension
143
Action Dimension
34
Learning Rate
1.0 x 10 -4
Discount Factor (r))
0.97
Tau (T)
0.01
Entropy Coefficient (ox)
0.2
Total Timesteps
1,500,000
Training Purameters
Batch Size
1024
Updates to Data Ratio
Learning Starts
300
Training Frequency
Policy Update Frequency
Target Update Frequency
Max Gradient Norm
Residual Scale
0.1
Residual Strategy
Progressive Exploration
10,000
Progressive Exploration Threshold
1,500
24

---

Reward Design. We carefully design the reward functions to guide the robotic manipulation policies through complex tasks. The reward functions for each task are as follows:
Grasp Task. The reward function for the grasping task encourages precise finger positioning and successful object lifting:
Tgrasp exp -5 max _d.-0.0,, + 100 max (0.2 - |Eurget 2current -0.01), (2)
where de is the distance from the i-th finger (thumb, index, middle) to the object center, 2target =
Astur + 0.2 is the target height, and Acurent the current object height.
Pour Task. The reward function for the pouring task guides the robot through grasping, lifting, and pouring:
pour = 5.0 T(task success) + 10 (Fgrasp_ + run) + 50 (rin + Tball bowll
where:
"grasp_die dist 0.5 exp(-8.00 (o00-8.0ddumm)) +exp( 1.00pp(88.--dmee
Min 50 max (0.08 - Meurrent 0.08), -000),
Tul 0.5 (1 Reup 8up).
Thall bowl = 10 exp (-5.0 max (dhall howl - 0)).
Lift Task. The reward function for the lift task encourages coordinated grasping and lifting, combin- ing the following components:
Tun Felli grasp Tright
Taync hhighh Po-
where:
Tsyne = 4xp max (Ssyn 0.2, 0)): Coordination reward based on the sum of average finger distances Ssyme = dhen + dright-
beight 10 min (max (A: 0), 1): Reward for lifting the object, where A is the change in object height (target: 0.15 m).
Pe min (5.0, -mm--0001 1(0max 30.0): Penalty for excessive tilt (threshold = 30%).
Tlet grasp: grapp Reward for left-hand grasping. based the average distance den between the left fingers and the object (exp(-8 max(die 0.08, 0))).
grasp- Reward for right-hand grasping, based on the average distance dright between the right fingers and the object (exp(-8- max max(dright 0.08, 0))).
Computing Resources. All experiments in residual policy training are conducted on single NVIDIA RTX 4090 GPU.
A.3 Data Collection and Environment Setup
This section details our progressive and controlled data collection strategy for generating diverse simulation scenarios. The strategy is structured as follows:
Progressive Data Collection Strategy. We employ systematic approach cover environment variations, object variations, and spatial variations in our simulation:
Environment Variations: We randomly sample environments from the available introduce diversity in background and lighting conditions settings.
Object Variations: We adopt a curriculum-bar based approach, starting with geometrically similar objects and gradually introducing more challenging ones to ensure smooth learning curve.
Spatial Variations: We begin generating spatial configurations near the source demonstration scene and progressively extend them more distant configurations within the manipulation workspace.
25

---

Each iteration of the data generation process covers broader range of variants and presents a higher difficulty level compared the previous one.
Scenario Sampling Strategy. To ensure comprehensive coverage of generalization factors (i.e., different objects, environments, spatial configurations), design scenario sampler. The sampler randomly samples scenarios from the entire set while guaranteeing that all factors are represented. For example, the third iteration of the pouring task, we sample from 1440 scenarios, and the sampler selects 125 scenarios that cover 12 objects, 12 environments, and 10 spatial configurations. The scenarios are centered around objects, with different environments and spatial combinations.
Trajectory Collection Strategy. We set the number of iterations to {1, 2..3. 3]. For each task, we generate 20, 100, and 500 trajectories in the three iterations, respectively. Each scenario is used to collect trajectories. We employ a finite mode for data collection, where in each scenario, we set the Try Time 10 and the Success Threshold to 4 successful trajectories. If the try time exceeds 10 and the number of successful trajectories is less than 4, the scenario flagged as failed, and we move to the next scenario. After collection, we downsample the target number of trajectories.
A.4 Evaluation Test Set and Success Rate Calculation Method
Evaluation Test Set. Our evaluation test set consists of two categories for each task:
To):): The object generalization test set for each round i. This set is designed to evaluate the model's performance on specific objects in given round.
Togr: The comprehensive test set for each task, which includes all objects from the To(i) sets across all rounds. The specific environments and spatial configurations for TOgP are detailed in the supplementary material (see the code provided).
For the (1) test sets, provide visualizations for each task to illustrate the object generalization scenarios. Below are the figures for each task: Figure 6, Figure 7, Figure 8, and Figure 9.
Success Rate Calculation Method. (1)Grasp Task. The success condition for the grasp task is defined as: the target object must be lifted to a height exceeding 20 cm. (2)Pour Task. The success condition for the pour task is determined by checking if any ball is inside the bowl. The key evaluation metric is: a ball considered inside the bowl if its horizontal distance to the bowl is less than em. (3) Lift Task. The success condition for the lift task is defined
the target object must be lifted to a height exceeding 15 em. (4) Handover Task. The success condition for the handover task is defined as: the target object must be lifted
height exceeding 15 em, with the right hand completely released (distance 15 cm) and the left hand maintaining a secure grasp (distance 10 cm) for 10 consecutive steps. (5) General Failure Condition. For all tasks except the handover task, if the execution time exceeds 600 steps without achieving the success condition, the task is deemed a failure. For the handover task, the maximum allowed execution steps are increased to 800.
2
= 3
Figure 8: Lift Task Evaluation Test Set (To (f)).
26

---

i =
i = 3
Figure 6: Grasp Task Evaluation Test Set (To(i)).
2
i= 3
Figure Pour Task Evaluation Test Set (To(.)..
i 2
3
Figure 9: Handover Task Evaluation Test Set (T))..
A.5 Performance Scaling and Extended Iteration Analysis
To examine the effect of further iterations, we evaluated DexFlyWheel performance up to iteration on the Grasp and Lift tasks:
27

---

Table 7: Extended iteration performance of DexFly Wheel. Success rates of policies on Grasp and Lift tasks across iterations. Values are reported as mean standard deviation, with the improvement over the previous iteration.
Iteration
Grasp SR (%)
Lift SR (%)
i= 1
15.0 + 2.1
13.9
i= 2
58.0 4.8 (+43.0)
4.64 4.630..5
i= 3
90.0 3.2 (+32.0)
7.39
i=4 4
92.5 (+2.5)
82.1 (+2.7)
i=5
93.2 2.5 (+0.7)
$ (+1.4)
These results indicate that iteration = 3 provides a practical trade-off between performance gain and computational cost. Performance continues improve in later iterations, demonstrating the potential of BexFlyWheel to further enhance data diversity with additional iterations.
A.6 Time Efficiency Comparison
Wall-clock Training Time. Table 8 reports the wall-clock time for training the DexFlyWheel policies. The base policy trained with IL requires 5h 40m, while the residual RL policy requires 6h 30m iteration. Across the full three-iteration DexFly Wheel process, total wall-clock training time is approximately hours. Note that the first iteration uses only IL.
Table Wall-clock Training Time, Wall-clock time required to train the base policy and residual policies for three DexFlyWhen Wheel iterations.
Policy Training (Iteration)
Wall-clock Time
Base policy (II.)
5h 40m
Residual policy (RI.)
6h 30m
Total (3 iterations)
30 hours
A.7 The Minor Adjustment Observation and Curriculum Learning Strategy
DexFlyWheel generalizes effectively to novel objects by leveraging the Minor Adjustment Observa- tion and curriculum-bate policy learning strategy. As introduced in the main text, we observe that manipulating different objects typically causes only minor changes in the manipulation trajectories. In this appendix, we first present experimental evidence supporting this observation and Based on these insights, we then describe our curriculum-basee policy training strategy, which progressively exposes the policy to more diverse objects to enhance generalization.
A.7.1 Experimental Validation of the Minor Adjustment Observation
Experimental Setup. To rigorously quantify the effect of object variations on manipulation trajec- tories, conducted controlled experiments along two axes: (1) varying object mass in dual-arm lifting task, and (2) varying object shape in a grasping task. Trajectory deviation is measured using JointDiff (mean absolute joint position difference in radians between the nominal trajectory and the adapted trajectory). We also track task success rates across iterations (i = 1, 3). To evaluate the subtlety of adjustments, we define the Residual Norm Ratio (RNR):
RNR
base
+
where at is the residual correction at timestep 1, abssc is the base action, and 10-6 prevents division by zero. Low RNR indicates minor adjustments rather than drastic changes.
Key Findings. Our experimental results provide strong evidence for the Minor Adjustment Obser- vation and the effectiveness of our curriculum strategy:
28

---

Table 9: Trajectory Difference under Varying Mass. JointDiff (radians) when the object mass is varied in a dual-arm Lift task.
Objects (Density)
JointDiff (rad)
Very Light Box (0.1)
0.23
Original Box (1)
Heavy Box (10)
0.74
Very Heavy Box (50)
1.06
Table 10: Trajectory Difference under Varying Shape. JointDiff (radians) when object shape is varied in Grasp task.
Objects
JointDiff (rad)
Tennis Ball (Default)
Bowling Ball
0.75
Coke Can
0.59
Eggplant
0.95
Water Bottle
1.18
Table 11: Policy Suecess Rates and Residual Norm Ratios. Success rates (SR) across curriculum iterations and average RNR for different objects.
Objects
SR i
SR 2
SR = 3
Very Light Box (0.1)
86.7
92.0
93.3
Heavy Box (10)
36.7
45.0
80.5
Very Heavy Box (50)
0.0
12.0
56.0
Tennis Ball (Default)
94.0
93.7
94.2
Bowling Ball
43.1
78.9
90.0
Coke Can
38.0
73.4
80.2
Eggplant
20.6
30.8
70.9
Cube
15.9
20.0
85.6
Average RNR (%)
14.3 # 2.6
16.5 # 2.1
Trajectory Adjustment Remain Modest: As shown in Tables 9 and 10, trajectory adjustment (JointDiff) remain relatively modest under reasonable object variations. While differences grow larger for extreme object properties (e.g., density 50.0 or highly non-spherical shapes like a water bottle), they generally indicate an adaptation rather than complete replanning of the trajectory.
Currieulum-Bart Based Training Substantially Improves Success Rates: Table 11 clearly demon- strates that our curriculum-base training strategy substantially improves success rates across all tested conditions, especially for initially challenging cases (e.g., eggplant, cube, very heavy objects). After three iterations, DexFlyWheel achieves strong performance even on objects that yielded very low success rates in early iterations. This highlights how the curriculum effectively guides the learning process to generalize to novel objects.
Low Residual Norm Ratio Confirms Minor Adjustments: The relatively low Average Residual Norm Ratio values (in the order of 10-2) confirm that the residual corrections generated by DexFlyWheel are indeed subtle adjustments to the base actions. This directly supports the Minor Adjustment Observation, indicating that the policy learns to finely adapt pre-existing trajectories rather than generating entirely ones from scratch for novel objects.
Scope and Limitations. The Minor Adjustment Observation holds for tasks where fundamental interaction modes remain consistent (e.g., grasping, lifting, pouring, handover). may not generalize to tasks requiring drastically different strategies, such as deformable object manipulation (e.g., cloth folding, knot tying) or precision assembly, where subtle changes object properties may require fundamentally different control strategies.
29

---

A.7.2 Curriculum-Based Based Policy Learning Strategy
Based on the Minor Adjustment Observation, we design a curriculum- based policy training strategy to progressively generalize to novel objects. Using the dual-arm lift task as an example:
Iteration Foundational Skills with a Simple Object. In the initial iteration (i 1), we begin training with a simple object. This foundational step allows the robot to acquire basic grasping and lifting skills simplified environment, establishing robust baseline for subsequent learning.
Iteration 2: Generalization to Geometry-Similar Objects. Following the foundational stage (i = 2), we introduce a set of objects that share geometric similarities with the initial training object (e.g., objects of similar primitive shapes but varying dimensions). By training on these geometry-similar objects, the robot starts to generalize its skills beyond the exact specifications of the simple box. During test-time evaluation, we observe that DexFly Wheel not only performs well on these seen objects but also demonstrates an initial level of generalization to unseen objects with different geometries within this category.
Iteration 3: Generalization Across Diverse Object Categories. In the third iteration (i 3), we expand the object diversity by incorporating various categories with different geometries and appearances. DexFlyWheen Wheel demonstrates improved generalization compared to earlier iterations, performing reliably on a wider range of previously unseen objects. This highlights the effectiveness of the curriculum-based strategy in supporting the robot's ability handle diverse object types.
30


Reviews:
[{"review_id": "u2Lb5u3HwZ", "reviewer": "Reviewer_ewWr", "summary": "This paper presents DexFlyWheel, a scalable framework for generating diverse and high-quality dexterous manipulation data in simulation. It starts from a single human demonstration, and then iteratively expands data diversity using a closed-loop pipeline that combines imitation learning, residual reinforcement learning, trajectory rollouts, and data augmentation. The key insight is to use imitation learning to preserve human-like behaviors and using RL and domain randomization to improve generalization. The system generates over 2,000 high-quality demonstrations per task and significantly improves policy success rates in both simulation and real-world evaluations.", "questions": "How does the performance of DexFlyWheel scale with the number of iterations? Is there a diminishing return beyond iteration 3?\n\nWhat was the wall-clock time and compute cost required to complete the three-iteration flywheel for one task?\n\nIs there a reason the real-world deployment only includes bimanual tasks? Would the single-arm grasp/pour tasks work as well?", "limitations": "As acknowledged by the authors, the main limitations are (1) reliance on hand-crafted reward functions for residual RL, and (2) lack of tactile sensing for contact-rich tasks. Additionally, DexFlyWheel has not yet demonstrated generalization across robot morphologies or sensor setups.", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "(+) The paper presents a compelling and modular pipeline that effectively combines imitation learning, residual RL, and augmentation. The architecture (Figure 2) is clear and reproducible.\n\n(+) The system achieves up to 81.9% success rate on challenging generalization test sets (including novel objects, environments, and spatial setups), and outperforms strong baselines (e.g., DexMimicGen) by 28.7% on average.\n\n(+) The authors validate the pipeline with real-world bimanual robot experiments, achieving high success rates on lift and handover tasks, which is a strong testament to the practicality of the proposed approach.\n\n(+) The contributions of the residual RL module and augmentation module are carefully disentangled through well-structured ablation studies (Table 1, Figure 5), and failure cases are systematically explored (Table 3).\n\n(-) The framework still requires manually defined task-specific reward functions for residual RL. This contradicts the goal of scaling with minimal human input and could limit generalization to new task families.\n\n(-) The system only relies on vision, object state, and proprioception. While well-justified for scalability, this might limit performance on fine-grained, contact-rich tasks that require force or tactile cues.\n\n(-) While Table 3 quantifies collection success rates, the paper lacks qualitative analysis or visualizations of typical failure modes during data generation or real-world execution.\n\n(-) It would be interesting to assess whether policies trained in one setting (e.g., robot hand or camera view) generalize to another, given the framework's goal of large-scale data generation.", "was_revised": true, "final_justification": "The rebuttal effectively addresses prior concerns with substantial new results: extended iteration scaling, compute/time analysis, single-arm real-world validation, qualitative failure cases, and cross-view/embodiment generalization. These additions improve both clarity and empirical support. The approach remains well-motivated, technically sound, and impactful for scalable dexterous manipulation data generation. I maintain my positive assessment and recommend acceptance.", "rebuttal": "# **Response to Reviewer ewWr**\nThank you so much for your thoughtful and constructive feedback. We are delighted that you found our method compelling, the experimental evaluation thorough, and the presentation clear! Below, we address the concerns you raised.\n\n---\n\n## **Q1. Performance scaling and diminishing returns**\n\n### **A1:**\n\nThe submission manuscript already consider this (Section 5.2, Table 1; the last row shows an average improvement rate of 396.4% across four tasks as the iteration number increases from 1 to 3). To provide a more detailed understanding of the performance trend, we conducted additional experiments and report the following:\n\n**(1) Performance scaling across iterations**\n\nWe introduce the *Marginal Improvement (MI)* metric based on the Success Rate of $\\pi_{\\text{combined}}$ on $T_{\\text{OEP}}$. \\\nMIᵢ = SRᵢ − SRᵢ₋₁.\n\n| Iteration | SR (%) | MI (%)       |\n|-----------|--------|--------------|\n| i = 1     | 16.5   | –            |\n| i = 2     | 43.9   | +27.4        |\n| i = 3     | 81.9   | +38.0        |\n\nWe observe consistent performance gains across iterations, with particularly strong improvement in iteration 3.\\\n**(2) Evaluation beyond iteration 3 (up to i = 5)**\n\n| Iteration | Grasp SR (%)         | Lift SR (%)          |\n|-----------|----------------------|----------------------|\n| i = 1     | 15.0 ± 2.1           | 13.9 ± 2.8           |\n| i = 2     | 58.0 ± 4.8 (+43.0%)  | 44.4 ± 4.6 (+30.5%)  |\n| i = 3     | 90.0 ± 3.2 (+32.0%)  | 79.4 ± 7.9 (+35.0%)  |\n| i = 4     | 92.5 ± 2.8 (+2.5%)   | 82.1 ± 6.5 (+2.7%)   |\n| i = 5     | 93.2 ± 2.5 (+0.7%)   | 83.5 ± 5.8 (+1.4%)   |\n\nThese results suggest that iteration *i = 3* is a practical balance between gain and cost. And the continued performance increase in later iterations highlights the potential of our framework to keep enhancing data diversity. We have added the extended results to Appendix A.5: Performance Scaling and Extended Iteration Analysis.\n\n---\n\n## **Q2. Wall-clock Time and Compute Cost**\n\n### **A2:**\n\nWe report the average wall-clock time and compute cost of completing the full three-iteration DexFlyWheel process across four tasks.\n\n**(1) Training Compute Cost:**\n\n- IL module: 8× NVIDIA A100 GPUs  \n- Residual RL: 1× NVIDIA RTX 4090 GPU\n\n**(2) Wall-clock Training Time:**\n\n| **Policy Training (Iter.)** | **Wall-clock Time** |\n|-----------------------------|----------------------|\n| Base policy (IL)            | 5h 40m               |\n| Residual policy (RL)        | 6h 30m               |\n| **Total (3 iterations)**    | ~30 hours            |\n\n*Note: the first iteration uses only IL.*\n\n**(3) Data Collection Time (500 trajectories):**\n\n| Method              | Time        |\n|---------------------|-------------|\n| Human Teleoperation | 12h 30m     |\n| DexMimicGen         | 4h 24m      |\n| DexFlyWheel (Ours)  | **2h 24m**  |\n\nThese results demonstrate the practicality and scalability of our method in generating large-scale, high-quality dexterous manipulation data with reduced human effort and modest compute requirements. We include these details in Appendix A.6: Training and Data Collection Wall-clock Time.\n\n---\n\n## **Q3. Is there a reason the real-world deployment only includes bi-manual tasks? Would the single-arm grasp/pour tasks work as well?**\n\n### **A3:**\n\nWe initially chose to showcase bi-manual tasks, as they better highlight the capabilities of DexFlyWheel in handling complex dexterous scenarios. Following your suggestion, we added real-world experiments for single-arm tasks and the baseline method following the same experimental setting in Section 5.5. All results have been updated in Section 5.5 Table 4.\n\n**Revised Section 5.5:**\n**Table 4. Real-world performance comparison across four tasks**\n\n| Method         | Single-arm Grasp (%) | Single-arm Pour (%) | Dual-arm Lift (%) | Dual-arm Handover (%) | Average (%) |\n|----------------|----------------------|----------------------|--------------------|------------------------|-------------|\n| DexmimicGen    | 70.5 ± 2.3           | 35.6 ± 3.1           | 59.1 ± 2.8         | 10.2 ± 1.9             | 43.85 ± 2.5 |\n| Ours           | 79.8 ± 1.7           | 76.4 ± 2.4           | 78.3 ± 1.9         | 63.3 ± 2.7             | 74.45 ± 2.2 |\n\nAs shown in Table 4, our method achieves robust performance in both single-arm and dual-arm tasks, especially on more complex dual-arm tasks (e.g., +53.1% on handover vs. baseline). This demonstrates the robustness and generalization ability of DexFlyWheel in real-world transfer.\n\n---\n\n## **W1. The framework still requires manually defined task-specific reward functions for residual RL.**\n\n### **A4:**\n\nAs shown in the Lift Task experiment below, while we use standard reward functions for efficiency, DexFlyWheel is compatible with sparse and LLM-driven reward setups.\n\n| Method                    | SR (%)       | Training Steps |\n|---------------------------|--------------|----------------|\n| Human-design rewards [1]  | **98.0 ± 2.1**   | **50w**            |\n| LLM-driven rewards [3]    | 80.9 ± 5.7   | 100w           |\n| Sparse reward             | 84.5 ± 3.6   | 200w           |\n\nFor many common dexterous manipulation tasks, well-established reward functions already exist and have been widely adopted [1]. Applying these mature reward formulations in DexFlyWheel is more efficient than sparse reward or LLM-driven rewards, due to its faster and more stable policy training **(2~4 times fewer training steps)**. There is our trade-off in minimal human input and efficient flywheel iterations. In the furture, we plan to explore how LLM-based reward generation methods [2–3] can be efficiently integrated into our framework.\n\n---\n\n## **W2. The system only relies on vision, object state, and proprioception. While well-justified for scalability, this might limit performance on fine-grained, contact-rich tasks that require force or tactile cues.**\n\n### **A5:**\n\nThank you for this question. Our framework is readily extendable to support additional modalities such as force or tactile signals. \nSpecifically, we have already explored obtaining contact force signals from the simulator’s physics engine to emulate tactile interactions. However, these signals often lack realism and do not align well with the noise profile of physical tactile sensors, limiting their direct utility. Given recent advances in visual-tactile policy learning [4], we plan to investigate the integration of realistic tactile signals, potentially via calibrated sensor models or sim-to-real sensor alignment, to further strengthen our framework’s applicability to fine-grained, contact-intensive tasks.\n\n---\n\n## **W3. Lack qualitative analysis or visualizations of failure cases.**\nWe have already provided visualizations of these failure modes in **our supplementary video (timestamp 1:45–2:04) and discussed representative failure cases in Section 5.4 (lines 303–305).**  Following your valuable suggestion, we have added Appendix A.7 to include snapshots and qualitative discussion of common failure scenarios in the revised manuscript.\n\n### **A6:**\n**Appendix A.7 Failure Cases Analysis** \\\n**(1) Failure Cases in Baseline:**\n\n1. Generalization to Novel Geometries: Replay-based method often fails on objects with different geometries (e.g., from a ball to bottles) due to the lack of exploration mechanism.  \n2. Dynamic Contact Sensitivity: In high-dynamics tasks (e.g., mid-air handovers), even small discrepancies in object rotation, timing, or hand trajectory can lead to unstable grasp or failed transfer.\n\n**(2) Failure Cases in Our Method:**\n\n1. IK Instabilities: In some dual-arm tasks, the solver occasionally hits singularities (e.g., elbow lockout), causing unstable or failed motions.\n2. Out-of-Workspace: Objects at edges result in unfeasible plans.\n\nDue to rebuttal limits, visualizations will be included in the revised manuscript.\n\n---\n\n## **W4. Do policies trained under one embodiment or camera view generalize to others?**\n\n### **A7:**\n\nOur dataset includes two robot hands (Psibot and Inspire) and multiple camera views (front, top, wrist). To assess generalization, we conducted two evaluations:\n\n(a) Cross-Viewpoint (Train: Front, Eval: Top):\n\n| Task  | Front View | Top View       |\n|-------|------------|----------------|\n| Grasp | 90.0 ± 3.2 | 32.3 ± 5.1     |\n| Pour  | 85.8 ± 3.5 | 45.3 ± 7.6     |\n\n(b) Cross-Embodiment (Train: Inspire, Eval: Psibot):\n\n| Task  | Inspire      | Psibot        |\n|-------|------------|----------------|\n| Grasp | 90.0 ± 3.2 | 40.6 ± 4.8     |\n| Pour  | 85.8 ± 3.5 | 38.2 ± 6.0     |\n\nDespite any optimization, policies show partial transfer, indicating the framework’s robustness. We agree this is a valuable direction and plan to explore joint cross-view and cross-embodiment training. We've included these results in revised manuscript as Appendix A.8: Evaluation of Cross-Viewpoint and Cross-Embodiment Generalization.\n\n---\n\n**References:**\n\n[1] Y. Chen et al., \"Bi-DexHands: Towards human-level bimanual dexterous manipulation,\" IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 46, no. 5, pp. 1-15, May 2024.\\\n[2] Y. J. Ma et al., \"Eureka: Human-level reward design via coding large language models,\" in Proc. Int. Conf. Learn. Representations (ICLR), Jan. 2024.\\\n[3] T. Xie et al., \"Text2Reward: Reward shaping with language models for reinforcement learning,\" in Proc. Int. Conf. Learn. Representations (ICLR), Jan. 2024.\\\n[4] H. Xue, \"Reactive Diffusion Policy: Slow-Fast Visual-Tactile Policy Learning for Contact-Rich Manipulation,\" in *Proc. Robotics: Science and Systems (RSS)*, 2025."}, {"review_id": "aRBeikRRVF", "reviewer": "Reviewer_KEGx", "summary": "This paper addresses the problem of how to generate data at-scale with limited human teleoperation time. The key insight of this paper is that manipulating different objects induces minor changes in trajectories. With this, the authors present DexFlyWheel, which is a continual learning and data collection pipeline to generate diverse and high-quality dexterous manipulation data from few actual teleoperation data. To evaluate DexFlyWheel, the authors compared it against DexMimicGen, another synthetic data generation method, and showed extensive improvement for four tasks.", "questions": "*  How much of this method is on manipulating objects that can be well modeled by simulator? The approach relies heavily on a simulator's ability to accurately capture the motions of the object upon interaction. The current experiments are on relatively easy-to-model objects. I may decrease the score if there is no or an uncompelling written justification. I may raise my score for a good written justification, I will raise my score for a good experimental results.\n\n* How widely applicable is this insight of \"manipulating different objects only induces minor changes in the manipulation\" break, for example, lifting geometrically similar but much heavier object? How does the approach respond to this? The current experiments are on relatively easy-to-model objects. I may decrease the score if there is no or an uncompelling written justification. I may raise my score for a good written justification, I will raise my score for a good experimental results.\n\n* Please fix typo: \"desgin\", numbering in baselines, there are two (4)s, \".(detailed analysis in Section 5.4.)\"", "limitations": "yes", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 4, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Positives\n* Very impressive task results across multiple tasks, compared to baselines (DexMimicGen)\n* Experimental setup is sound, covering expected baselines\n* Approach is sound and intuitive (leveraging a continual learning framework while learning a residual policy to handle the slight differences seems to be a good way to decompose the problem)\n\nAreas of improvement\n* Comment and experiments showing if this is limited to manipulating things that can be well modeled by simulator? The approach relies heavily on a simulator's ability to accurately capture the motions of the object upon interaction. The current experiments are on relatively easy-to-model objects. \n* Comment and experiments on where this insight of \"manipulating different objects only induces minor changes in the manipulation\" break, for example, lifting geometrically similar but much heavier object.\n* Typo: \"desgin\", numbering in baselines, there are two (4)s, \".(detailed analysis in Section 5.4.)\"", "was_revised": true, "final_justification": "Thank you to the nice rebuttal by the authors. My questions and issues have been resolved, and I appreciated the additional experiments taken to stress test the approach.\nI have raised the sub-scores accordingly.", "rebuttal": "# **Response to Reviewer KEGx**\nWe are so grateful for such a detailed review and positive evaluation. It is encouraging to receive recognition for our core ideas, impressive performance and clear writing! We sincerely appreciate your efforts to help us improve the quality of our work. Below, we address the concerns you raised.\n\n---\n\n## **Q1. How much of this method is on manipulating objects that can be well modeled by simulator? The approach relies heavily on a simulator's ability to accurately capture the motions of the object upon interaction. The current experiments are on relatively easy-to-model objects.**\n\n### **A1: Justification for Simulation Dependence**\n\nThank you for raising this important point. Like all simulation-based data generation methods [1-3], our approach relies on the simulator’s ability to model object interactions. This is a common and necessary trade-off that enables scalable and human-effort-less data collection compared to real-world alternatives. Thanks to recent advances [4-6], modern simulators now support complex physics and contact dynamics, and have been widely used in impactful large-scale data generation works [1-3], contributing significantly to the community.\n\nWe acknowledge that simulation still has limitations in modeling fine-grained physical interactions. However, this gap is narrowing rapidly with the development of high-fidelity physics engines like physX. We have added a note in the revised manuscript’s Limitations section to explicitly acknowledge this.\n\nImportantly, we also include results on more complex objects and demonstrate transferability to real hardware when the simulated dynamics are imperfect—highlighting the robustness of our method in such cases.\\\n**1. Experiment: Articulated Object Manipulation Task**\n- **Setup:** Task involves opening a 1-DOF articulated door.\n\n| Task         | Iteration | O | E | P | Configs | Traj. | SR in TOEP (%) |\n|--------------|-----------|---|---|---|---------|-------|----------------|\n| Open a Door  | i=1       | 1 | 1 | 1 | 1       | 20    | 34.2           |\n|              | i=2       | 2 | 5 | 5 | 50      | 100   | 70.6           |\n\nThis demonstrates that DexFlyWheel effectively handles more complex objects. \\\n **2. Experiment: Robustness to Mass Variation (Lift Task)**\n- **Setup:** Trained on the simulated \"Original\" box (density = 1.0). Evaluated on both unseen simulated and real-world \"Heavy\" boxes (density = 10.0). The geometry and appearance are identical.\n\n| Iteration           | Sim (Original) | Sim (Heavy) | Real (Original) | Real (Heavy) |\n|---------------------|----------------|-------------|-----------------|--------------|\n| Iter 1     | 86.7           | 36.7        | 70.2            | 10.3         |\n| Iter 3 | 93.3      | 80.5        | 73.8            | 70.7         |\n\n- **Key Findings:**  \n  1. The residual RL module adapts well to differences in dynamics, improving generalization to heavy objects (Iteration 3).  \n  2. Real-world transfer succeeds despite training-simulation mismatch, due to diverse data generated across iterations. This shows that DexFlyWheel can handle a certain degree of inaccuracies in dynamics modeling.\n\nDexFlyWheel mitigates simulation inaccuracies by generating diverse data through residual RL and augmentation. Residual RL exposes the policy to objects with varying physical properties (e.g., mass, friction), while augmentation diversifies environmental appearance and object poses, encouraging generalization beyond simulator specifics.\n\n**Revisions**\n- We have added the results of the articulated object manipulation task in Appendix A.9: Additional Tasks Performance.\n- We have included the evaluation of simulation-to-real dynamics consistency in Appendix A.10: Robustness to Simulation Inaccuracies in Object Dynamics.\n\n\nWe appreciate the opportunity to clarify our method’s reliance on simulation fidelity and mitigation strategies. In future work, we plan to incorporate advances in sim-to-real transfer to further enhance robustness, and extend DexFlyWheel to more complex domains such as fluids and deformable objects [7].\n\n## **Q2. How widely applicable is this insight of \"manipulating different objects only induces minor changes in the manipulation\"?**\n\n### **A2. Experiments and Justification for the Minor Adjustment Hypothesis**\n\nThank you for your insightful question! This core observation directly corresponds to the key insight behind our iterative, curriculum-based learning approach.\n\nThis observation holds robustly when object variations fall within plausible ranges, while in extreme cases (e.g., very large density or shape differences), trajectory deviations become larger. To handle this, our method employs a curriculum mechanism that progressively introduces object diversity across iterations, enabling effective policy adaptation. As described in Sec. 5.1 and the caption of Fig. 3(b): *\"Object diversity is expanded across iterations, from a single object (i=1), to geometrically similar objects (i=2), and finally to objects with diverse geometries and physical properties (i=3).\"* \n\nTo validate our hypothesis that object variations induce only minor trajectory adjustments, we conducted controlled experiments across two axes: (1) object mass (lift task), and (2) object shape (grasp task). We quantified trajectory deviation via JointDiff, and report success rates (SR) across curriculum iterations (i=1,2,3) alongside Residual Norm Ratio (RNR),  which measures the relative magnitude of residual corrections compared to base policy actions.\n\n**Experiment: Trajectory Difference under Varying Mass and Shape**\n\n**Table 1.1: Trajectory Difference under Varying Mass (Lift Task)**\n\n| Density (Fixed Type) | JointDiff (rad) |\n|----------------------|-----------------|\n| 0.1 (Very Light)     | 0.23        |\n| 1.0 (Original)       | –               |\n| 10.0 (Heavy)         | 0.74        |\n| 50.0 (V-Heavy)       | 1.06        |\n\n **Table 1.2: Trajectory Difference under Varying Shape (Grasp Task)**\n\n| Object         | JointDiff (rad) |\n|----------------|-----------------|\n| Tennis Ball    | –               |\n| Bowling Ball   | 0.75           |\n| Coke Can       | 0.59           |\n| Eggplant       | 0.95           |\n| Water Bottle   | 1.18           |\n\n**Table 2: Success Rates of DexFlyWheel Policy Across Iterations and Residual Norm Ratio**\n| Condition                     | SR i=1 (%) | SR i=2 (%) | SR i=3 (%)     |\n| ----------------------------- | ---------- | ---------- | -------------- |\n| Density 1.0 (Original)        | 86.7       | 92.0       | 93.3           |\n| Density 10.0 (Heavy)          | 36.7       | 45.0       | 80.5           |\n| Density 50.0 (V-Heavy)        | 0.0        | 12.0       | 56.0           |\n| Tennis Ball (Default)         | 94.0       | 93.7       | 94.2           |\n| Bowling Ball                  | 43.1       | 78.9       | 90.0           |\n| Coke Can                      | 38.0       | 73.4       | 80.2           |\n| Eggplant                      | 20.6       | 30.8       | 70.9           |\n| Cube                          | 15.9       | 20.0       | 85.6           |\n| Residual Norm Ratio (RNR)|     -      | 14.3 ± 2.6 |16.5 ± 2.1 |\n\nRNR is computed as:\n\n$$\n\\mathrm{RNR} = \\frac{1}{T} \\sum_{t=1}^T \\frac{ \\| r_t \\| }{ \\| b_t \\| + \\epsilon }\n$$\n\nwhere \\$r_t$ is the residual action at time step $t$, $b_t$ is the base action, and $\\epsilon$ is a small constant to prevent division by zero.\n\n **Key Findings:**\n- Trajectory deviations remain modest for reasonable object variations but increase with extreme differences.\n- Success rates improve significantly over curriculum iterations, demonstrating effective policy adaptation.\n- The relatively low Residual Norm Ratio indicates that only subtle residual corrections are necessary.\n- Our curriculum design enables robust generalization across both shape and physical property variations.\n\nWe hope our controlled experiments and detailed comparisons have addressed your concerns. Thank you again for your effort and for raising such insightful questions.\n\n## **Q3. Please fix typo.**\n> **A3**: According to your valuable suggestions, we have carefully checked and corrected the errors:\n> - **Typo**: We have changed \"desgin\" to \"design\" (Sec. 5, Line 199).\n> - **Duplicate numbering**: We have renumbered the baselines correctly in Sec. 5.1 (Lines 246–249), where the duplicate \"(4)\" has been fixed.\n\n\n**References:** \\\n[1] Mandlekar, A., et al. \"MimicGen: A Data Generation System for Scalable Robot Learning using Human Demonstrations\", CoRL 2023.  \\\n[2] Wang, R., et al. \"DexGraspNet: A Large-Scale Robotic Dexterous Grasp Dataset for General Objects Based on Simulation\", ICRA 2023.  \\\n[3] Jiang, Z., et al. \"DexMimicGen: Automated Data Generation for Bimanual Dexterous Manipulation via Imitation Learning\", ICRA 2025.  \\\n[4] Li, C., et al. \"BEHAVIOR-1K: A Human-Centered, Embodied AI Benchmark with 1,000 Everyday Activities and Realistic Simulation\", arXiv preprint arXiv:2403.09227, 2024.  \\\n[5] Tao, S., Xiang, F., Shukla, A., et al. \"ManiSkill3: GPU Parallelized Robotics Simulation and Rendering for Generalizable Embodied AI\", arXiv preprint arXiv:2410.00123, revised May 2025.  \\\n[6] Todorov, E., Erez, T., and Tassa, Y. \"MuJoCo: A Physics Engine for Model-Based Control\", IEEE/RSJ IROS, 2012, pp. 5026–5033. doi: 10.1109/IROS.2012.6386109.  \\\n[7] Wang, Y., et al. “DexGarmentLab: Dexterous Garment Manipulation Environment with Generalizable Policy”, arXiv:2505.11032, 2025."}, {"review_id": "PITNcrvKTU", "reviewer": "Reviewer_5Q2M", "summary": "The paper introduces DexFlyWheel, a scalable framework for generating high-quality and diverse dexterous manipulation datasets from minimal human demonstrations. It combines imitation learning (IL) and residual reinforcement learning (RL) in a self-improving cycle. Starting from a single human demonstration per task, DexFlyWheel iteratively augments data diversity and policy generalization by rolling out trajectories in simulation, applying domain randomization, and refining policies via RL. The method is validated on multiple dexterous manipulation tasks (grasp, pour, lift, handover) and achieves strong performance both in simulation and real-world deployment", "questions": "- Why were only the dual-arm tasks (Lift, Handover) deployed in real-world experiments? Were there specific challenges in deploying single-arm tasks (Grasp, Pour)? Including even preliminary hardware results for single-arm tasks would strengthen claims of broad applicability.\n- The absence of real-world deployment for baseline methods limits comparative insights into sim-to-real performance. Could the authors discuss any technical or fairness challenges that prevented such evaluation? Would they consider adding such comparisons in future work or supplementary material?\n- The current evaluation uses four rigid-object tabletop tasks. Can the authors comment on the feasibility of extending DexFlyWheel to tasks with articulated objects, or long-horizon manipulation? This would provide a more robust test of the framework’s scalability.", "limitations": "Yes", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 4, "originality": 3, "strengths_and_weaknesses": "Strengths:\n- Novelty: Introduces a closed-loop “FlyWheel” mechanism integrating IL and RL, enabling scalable data generation from minimal seed demonstrations.\n- Highly Scalable: Achieves over 2000 diverse demonstrations from just one teleoperated demo per task.\n- Extensive simulation experiments with multiple baselines and ablation studies.\n- Demonstrates promising sim-to-real transfer on dual-arm Lift tasks.\n\nWeakness:\n- While the simulation comparisons are thorough, it would significantly strengthen the claims of superior real-world generalization if baseline methods used in simulation were also deployed on hardware. This would help quantify the sim-to-real gap across methods and ensure fair evaluation.\n- While the real-world deployment of DexFlyWheel on dual-arm tasks demonstrates promising sim-to-real transfer, no real hardware results are reported for single-arm tasks (Grasp and Pour). This omission limits the ability to fully assess the framework’s generalization performance across all studied tasks.\n- The framework’s evaluation covers 4 dexterous manipulation tasks, which provides initial evidence of scalability. However, these tasks are limited. It would be helpful to add more tasks.", "was_revised": true, "final_justification": "I have read the rebuttal the authors posted, which addressed my questions and concerns.", "rebuttal": "# **Response to Reviewer 5Q2M**  \nWe sincerely thank you for the thoughtful and encouraging feedback. We are glad that you recognized the novelty of our closed-loop DexFlyWheel framework, its significance for robotics community and its highly scalable for data generatoin. Below, we address your additional questions and suggestions in detail.\n\n---\n\n## **Q1/Q2/W1/W2. Simulation Baselines and Single-Arm Task Real-World Results in Real-world**\n\n### **A1:**  \nThank you for pointing this out. In the submission manusricpt, we chose to showcase real-world deployment on more challenging bimanual tasks, as they better highlight the capabilities of DexFlyWheel in handling complex dexterous scenarios. As suggested, we extended our evaluation to include (1) single-arm tasks (Grasp and Pour) and (2) real-world deployments of the top-performing baseline method (DexmimicGen). To ensure a fair comparison, all methods were deployed following an identical pipeline, as described in Section 5.5.\n\n| Method       | Single-arm Grasp (%) | Single-arm Pour (%) | Dual-arm Lift (%) | Dual-arm Handover (%) | Average (%) |\n|--------------|---------------------|--------------------|-------------------|----------------------|-------------|\n| Ours         | 79.8                | 76.4               | 78.3              | 63.3                 | 74.45       |\n| DexmimicGen  | 70.5                | 35.6               | 59.1              | 10.2                 | 43.85       |\n\nThe results show that our method demonstrates strong performance on both single- and dual-arm tasks, confirming its broad applicability. Furthermore, it significantly outperforms the baselines across all real-world tasks, highlighting its superior potential for practical application.\n\n**Revisions:**  \nWe have incorporated these new findings into the revised manuscript in Sec. 5.5, featuring a new \"Table 4: Real-World Deployment Results with Baselines.\" These results provide direct evidence of our method's robustness and effectiveness in practical scenarios.\n\n\nHope the above responses address your concerns. Thank you again for your time and effort!\n\n## **Q3/W3. The current evaluation uses four rigid-object tabletop tasks. Can the authors comment on the feasibility of extending DexFlyWheel to tasks with articulated objects, or long-horizon manipulation?**\n\n### **A2:**  \nWe thank the reviewer for this question regarding the scalability of our framework. Conceptually, our framework is designed with scalability in mind. It is task-general by design: the VR interface allows flexible teleoperation for collecting demonstrations across diverse tasks, and the combination of imitation learning and residual reinforcement learning operates on standard inputs (e.g., RGB images, object state, and robot state) that are applicable to a wide range of manipulation scenarios.\n\nFollowing your valuable suggestion, we have conducted new experiments to test the feasibility of extending DexFlyWheel to more complex scenarios, specifically articulated object manipulation (opening a door).\n\n| Task      | Iter. | O | E | P  | Configs | Traj. | SR in TOEP (%) |\n|-----------|-------|---|---|----|---------|-------|----------------|\n| Open Door | i=1   | 1 | 1 | 1  | 1       | 20    | 34.2           |\n|           | i=2   | 2 | 5 | 5  | 50      | 100   | **70.6**           |\n\nOur preliminary results from these new experiments are encouraging. They demonstrate that DexFlyWheel can effectively scale to more complex tasks.\n\nRegarding task horizon, our existing Pour and Handover tasks already exhibit a certain degree long-horizon characteristics. The Pour task requires a two-stage pick-and-pour sequence, while Handover involves coordinated bimanual manipulation, including object pickup and hand-to-hand transfer (Sec. 5.1, L208–209 & L211–212).\n\nTo further evaluate long-horizon scalability, we conducted an additional three-stage task: Pick-Pour-Place, where the robot must (1) pick up a container, (2) pour its contents into a target receptacle, and (3) place the container at a designated location. This task requires maintaining precise control across multiple object poses and transitions:\n\n| Task            | Iter. | O | E | P | Configs | Traj. | SR in TOEP (%) |\n|-----------------|-------|---|---|---|---------|-------|----------------|\n| Pick-Pour-Place | i=1   | 1 | 1 | 1 | 1       | 20    | 30.4           |\n|                 | i=2   | 2 | 3 | 3 | 18      | 100   | 65.7           |\n\nThe improvement in success rate confirms that our framework can also accommodate longer-horizon, multi-stage manipulation tasks.\n\nWe view extending DexFlyWheel to longer-horizon, multi-stage manipulation tasks as an important and practical next step. In future work, we plan to integrate hierarchical policy learning into our residual framework, enabling temporal abstraction and skill composition.\n\n---\nHope the above clarifications and responses address your concerns. Thank you again for your time and effort!"}, {"review_id": "9yIoXRMTSQ", "reviewer": "Reviewer_XGgH", "summary": "This paper proposes a data flywheel approach to training robots in simulation. A VR teleoperation interface collects a small set of human demonstrations that are used to train a behavior cloning policy. This base policy is then modified with a residual policy using reinforcement learning in a simulated setup. Data collected with this modified policy is filtered, augmented, and used to improve the base policy using imitation learning. Results show iterative improvement as cycles of the flywheel are run, including transfer to real robot setups.", "questions": "- If DexFlyWheel requires simulators and rewards, how is this different and better than other RL at scale works like MT-Opt (https://arxiv.org/pdf/2104.08212) that also have a filtered data loop?\n- Why are initial success rates at iter i = 1 (Table 2) higher than the initial SR in iters i = 2 and i = 3? The near 100% success rates feel odd given that the policy has only been trained on 20 trajectories, especially because the robot is using  What can explain the high initial success rates and the initial ability for the robot to learn from 20 trajectories only?\n- What exactly is your data augmentation module A_EP doing? It is briefly mentioned in lines 149-155 but it seems like a contribution in the pipeline, so it is worth discussing in more detail if possible.\n\nI’d be willing to increase the scores and overall rating if the authors present a compelling argument on the originality of this approach and its advantages over other continual RL/IL paradigms, as well as clarified why TO^i results start high and decrease.", "limitations": "- Specify the need for simulations for this pipeline to work (given the amount of residual RL and rewards needed)\n- Reduce the strength of the paper claim: “infinite diverse data generation only from one demonstration per task”", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 2, "strengths_and_weaknesses": "Strengths \n\n- VR teleoperation with bimanual hand tracking makes human data collection relatively straightforward\n- meaningful results with sim2real transfer\n- results show improvement in generalization tests with each iteration of the flywheel\n\nWeaknesses \n\n- although this approach requires comparatively few human demonstrations, it also requires access to ground truth simulation reward and carefully engineered reward functions\n- Requires simulation and carefully engineered rewards to work with the RL part\n- The strong claim of “infinite diverse data generation only from one demonstration per task” does not match the experiments, which collect 20 trajectories per task in the first iteration. It is an overclaim.\n- Performance drop in T_O^i (Table 2) with added data does not make sense and not much explanation is given about this observation. It is especially surprising given that the tasks require dexterous manipulation.\n- filtering requires success detection, which is accessible in simulation but not in the real world without additional modues\n- There exist a growing body of work focusing on data flywheel-type approaches, but there are limited mentions of these works. These works below may not all be fully relevant, but provided here for convenience\n    - https://arxiv.org/pdf/2411.01813v1\n    - https://arxiv.org/abs/2405.10315\n    - https://arxiv.org/pdf/2407.16677v1\n    - https://arxiv.org/pdf/2505.10442v1\n    - https://proceedings.mlr.press/v164/lu22a/lu22a.pdf\n    - https://arxiv.org/pdf/2410.14868v4", "was_revised": true, "final_justification": "See response to author rebuttal for the full details. Nearly all of my concerns from the original reviews were original misunderstandings (clarified by the authors) or addressed by additional experiments. Rebuttal remarks about DexFlywheel's scope and novelty were also convincing. I would appreciate an increase of paper clarify in the final publication to dispel similar misunderstandings by other readers.", "rebuttal": "# **Response to Reviewer XGgH**  \nWe sincerely appreciate your thoughtful summary and recognition of our work’s strengths. Below, we address the concerns you raised.\n\n## **Clarifying Our Work’s Focus**\nWhile you characterized our method as a “policy training approach,” we would like to clarify that DexFlyWheel’s primary contribution is a scalable, data-centric framework for generating diverse, high-quality dexterous manipulation data, rather than a new policy learning algorithm. This distinction is crucial for understanding our work’s novelty and scope.\n\n---\n\n## **Q1. If DexFlyWheel requires simulators and rewards, how is it different and better than other large-scale RL works like MT-Opt that also use filtered data loops?**\n\n### **A1. Different Goals and Lightweight Residual RL Module**  \nOur goal fundamentally differs from that of RL algorithm papers such as MT-Opt. While MT-Opt focuses on multi-task RL policy design for gripper-based tasks, DexFlyWheel focuses on building a scalable data generation framework for dexterous manipulation involving high-DOF hands and complex tasks.\n\nAdditionally, our residual RL module is lightweight and efficient, used to *refine* base policies trained via imitation learning (IL) to improve generalization while preserving human-like behavior. This contrasts with MT-Opt’s more complex RL modules focused on large-scale policy learning.\n\n## **Q2/W4. Why are initial success rates at iteration i=1 (Table 2) higher than at i=2 and i=3? How can the policy perform well with only ~20 trajectories?**\n\n### **A2. Increasing Test Set Difficulty Across Iterations**  \n$T_O^i$ is not a fixed test set—each round introduces harder objects and more diverse scenes, making later evaluations progressively more challenging (Sec. 5.1, L233–237). To clarify this, we added a new experiment on the Lift task, where we evaluate policies from each iteration on all test sets ($T_O^1$: easy, $T_O^2$: medium, $T_O^3$: hard):\n\n| Task  | Iter. | $T_O^1$ (Easy) | $T_O^2$ (Medium) | $T_O^3$ (Hard) |\n|-------|-------|--------------------|---------------------|--------------------|\n| Lift  | 1     | 90.0 ± 2.2        | 45.0 ± 3.1         | 18.9 ± 2.5        |\n|       | 2     | 93.0 ± 3.5        | 83.3 ± 3.8         | 59.0 ± 6.9        |\n|       | 3     | 93.4 ± 2.6        | 90.3 ± 1.5         | 98.0 ± 2.1        |\n\nThese results confirm continuous improvement on increasingly challenging test sets as the flywheel iterates.\nWe have renamed $T_O^i$ to $TO(i)$ in the revised manuscript for clarity (Sec. 5.1, L233–237, Table 1).\n\n---\n\n## **Q3. What exactly does the data augmentation module $A_{EP}$ do? It’s briefly mentioned but seems important.**\n\n### **A3. Multi-Level Scene Augmentation to Enrich Data Diversity**  \nThe  $A_{EP}$ module applies multi-level augmentations during warm-up and flywheel phases:\n- Environment-Level Enhancements:\n  - Lighting variation (e.g., ±20% brightness, color temperature shifts)\n  - Workspace appearance changes (e.g., wood to metal surfaces)\n- Spatial-Level Enhancements:\n  - Object Pose Variation to object initial poses (e.g., ±5 cm translation)\n\nThis module is first applied to seed demonstrations to bootstrap a diverse distribution of initial trajectories (20 demos), and it continues to promote data diversity during the iterative flywheel phase.\n\nIn the revised manuscript (Section 4.3, lines 149–151), we have updated the description of the data augmentation module $A_{EP}$ to provide a more comprehensive clarification of the above.\n## **Q4. Comparison to continual IL/RL approaches (which in the reviewer questions summary)**\n\n### **A4. Hybrid IL + Residual RL Framework for Efficient, Human-like and Diverse Data Generation**\nThank you for the insightful question!  This is a critical point that gets to the heart of our core design: \n- Imitation Learning (IL) provides human-like behavior priors and significantly accelerates the exploration process in reinforcement learning.\n- Residual Reinforcement Learning (Residual RL) efficiently refines the base policy learned via IL, improving generalization and enabling diverse, high-quality data generation.\n\nIn contrast:\n- Pure IL tends to suffer from poor generalization to novel objects or scenarios.\n- Pure RL often produces unnatural or low-quality trajectories and suffers from low exploration efficiency.\n\nWe have conducted experiments to validate these claims.\n\n**Compared to continual pure IL:**\n\n- In our manuscript, experiments (see Fig. 5) already confirm that continual pure IL struggles to generalize to novel objects. Specifically, it achieves only 8.95% success on the challenging test set—a 72.95% drop compared to DexFlyWheel. Even with strong augmentations, the performance improves only marginally to 18.9%, highlighting the limitations of continual pure IL in generalization.\n\n**Compared to continual pure RL:**\n\nWe ran an additional Lift task experiment (from iteration 2 to 3) comparing continual pure RL and our method:\n\n| Method           | SR  i=3 ↑ | Training Time ↓ | Trajectory Jerkiness ↓ |\n|------------------|-------------|-----------------|-----------------------|\n| Continual Pure RL | 43.8 ± 5.7 | 20 hours        | 2.07                  |\n| Ours             | 98.0 ± 2.1 | 6.5 hours       | 0.28                  |\n\n- Performance: Our approach achieves a significantly higher SR (98.0 vs. 43.8) due to the strong IL-based initialization.  \n- Efficiency: Continual RL must explore from scratch (20h), while our method benefits from the base IL policy, reducing RL search space (6.5h).  \n- Trajectory Quality: Continual Pure RL often generates jerky, non-human-like motions. Our method preserves smooth and natural behavior (0.28 vs.2.07).\n\n## **W1/W2. Although your method needs few human demos, it requires ground truth simulation rewards and careful reward design.**\n### **A5. Compatibility with Sparse and LLM-Driven Rewards**\n\nWe demonstrate that DexFlyWheel is compatible with sparse and LLM-driven rewards (Lift task):\n\n| Reward Type           | Success Rate (%) | Training Steps    |\n|----------------------|------------------|-------------------|\n| Human-designed [1]    | 98.0 ± 2.1       | 0.5M              |\n| LLM-driven [2]       | 80.9 ± 5.7       | 1.0M              |\n| Sparse               | 84.5 ± 3.6       | 2.0M              |\n\nMany mature reward functions for common dexterous manipulation tasks have already been widely adopted in the community [1]. Applying these well-established reward formulations in DexFlyWheel proves more efficient than using sparse or LLM-driven rewards [2], due to faster convergence and more stable policy training (2~4 times fewer training steps).\n\nIn future work, we plan to explore how LLM-driven reward generation methods can be efficiently integrated into DexFlyWheel.\n\n## **W3. The claim of “infinite diverse data from one demonstration” conflicts with collecting 20 demo trajectories initially.**\n### **A6:**\nWe respectfully clarify your misunderstanding: our method relies on only a single human demonstration per task, as described in Section 4.2 (Line 146). The “20 trajectories” reported in Table 1 are automatically generated through our data augmentation module from that single demo, with no further demonstration input.\n\n## **W5. Filtering requires success detection, which is accessible in simulation but not in the real world without additional modues.**\n\n### **A7:**\nWe would like to clarify that our approach is a simulation-based data generation framework, which does not require collecting or filtering data in the real world. This is a common design choice adopted in many representative simulation-based data generation methods [3–5].\n\n## **W6. There exist a growing body of work focusing on data flywheel-type approaches, but there are limited mentions of these works. These works below may not all be fully relevant, but provided here for convenience**\n\n### **A8:**\nWhile several recent works have explored IL/RL-based flywheel concepts, they primarily focus on gripper-based tasks and emphasize policy tuning or online correction rather than data generation itself. In contrast, DexFlyWheel is specifically designed for dexterous manipulation involving high-DOF hands and complex tasks, and it serves as a complete data-centric framework with a principled flywheel design that drives scalable data generation and policy improvement.\n\n\n### **Revisions**\n- **Related Work:** Expand discussion of continual IL/RL and flywheel-style frameworks, clarifying DexFlyWheel’s distinct contributions.  \n- **Method:** Extended the description of the $A_{EP}$ module to highlight its role in generalizing across task variations through scene randomization and trajectory replay.  \n- **Notation:** Rename $T_O^i$ to $TO(i)$ for clarity.\n\n---\n\nWe appreciate your valuable feedback, which improved our paper’s clarity. Other reviewers recognize the novelty of our closed-loop flywheel, scalability (2000+ trajectories), and real-world effectiveness (78.3% dual-arm lift SR). We hope our detailed responses demonstrate DexFlyWheel’s significance as the first policy-in-the-loop data generation framework for dexterous manipulation, and respectfully request reconsideration of your rating.\n\n---\n\n\n**References:** \\\n[1] Bi-DexHands: Towards human-level bimanual dexterous manipulation IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 46, no. 5, pp. 1-15, May 2024.\\\n[2] Eureka: Human-level reward design via coding large language models,\" in Proc. Int. Conf. Learn. Representations (ICLR), Jan. 2024.\\\n[3] A. Mandlekar et al., “MimicGen: A Data Generation System for Scalable Robot Learning using Human Demonstrations,” CoRL, 2023. \\\n[4] R. Wang et al., “DexGraspNet: A Large-Scale Robotic Dexterous Grasp Dataset for General Objects Based on Simulation,” ICRA, 2023. \\\n[5] Z. Jiang et al., “DexMimicGen: Automated Data Generation for Bimanual Dexterous Manipulation via Imitation Learning,” ICRA, 2025."}]

Author rebuttal:

Meta-review: This paper presents a scalable approach for generating diverse and high quality demonstrations in simulation for dexterous manipulation tasks. The pipeline can start with as little as a single human demonstration and through a self-improving cycle, it can gradually increase the high quality demonstrations to up to 2000. Initially, the paper received one Reject, one Borderline Accept and two Accept ratings. The reviewers were impressed with the performance of the proposed pipeline compared to previous work, its scalability and practicality. However there were some remaining questions, e.g., related to scope, comparison with prior work, reliance on manually defined task-specific reward functions and limitations imposed by the capabilities of the simulator. During the rebuttal period, the authors replied to the concerns of the reviewers, with additional experiments and clarifications. The rebuttal answers were received very well by the reviewers and eventually they increased their ratings to one Borderline Accept and three Accept ratings. Given the strong and unanimous support from four knowledgable reviewers, there is no basis to overturn reviews. This is a well-motivated approach with strong results that has the potential to be quite impactful. The AC recommends acceptance. Authors should still consider any additional comments or feedback from the reviewers while preparing their final version (e.g., improve clarity), and of course update the manuscript to include any additional promised changes, analysis, results and/or discussion they provided in their rebuttal.
Decision: Accept (spotlight)