Compliant Residual DAgger: Improving Real-World Contact-Rich Manipulation with Human Corrections
Authors: Xiaomeng Xu
Yifan Hou
Zeyi Liu
Shuran Song
Venue: NeurIPS 2025 poster ()
Keywords: robot manipulation
imitation learning
human-in-the-loop learning
Primary area: applications
TL;DR: A system that improves real-world contact-rich robot manipulation policies with human corrections

Abstract: We address key challenges in Dataset Aggregation (DAgger) for real-world contact-
rich manipulation: how to collect informative human correction data and how to
effectively update policies with this new data. We introduce Compliant Residual
DAgger (CR-DAgger), which contains two novel components: 1) a Compliant
Intervention Interface that leverages compliance control, allowing humans to pro-
vide gentle, accurate delta action corrections without interrupting the ongoing
robot policy execution; and 2) a Compliant Residual Policy formulation that learns
from human corrections while incorporating force feedback and force control.
Our system significantly enhances performance on precise contact-rich manipu-
lation tasks using minimal correction data, improving base policy success rates
by over 60% on two challenging tasks (book flipping and belt assembly) while
outperforming both retraining-from-scratch and finetuning approaches. Through
extensive real-world experiments, we provide practical guidance for implementing
effective DAgger in real-world robot learning tasks.

Compliant Residual DAgger: Improving Real-World Contact-Rich Manipulation with Human Corrections
Xiaomeng Xu"
Yifan Hou"
Zeyi Liu
Shuran Song
Stanford University
Abstract
We address key challenges Dataset Aggregation (DAgger) for real-world contact- rich manipulation: how to collect informative human correction data and how to effectively update policies with this new data. We introduce Compliant Residual DAgger (CR-DAgger), which contains two novel components: 1) Compliant Intervention Interface that leverages compliance control, allowing humans to pro- vide gentle, accurate delta action corrections without interrupting the ongoing robot policy execution; and 2) Compliant Residual Policy formulation that learns from human corrections while incorporating force feedback and force control. Our system significantly enhances performance on precise contact-rich manipu- lation tasks using minimal correction data, improving base policy success rates by over 60% on two challenging tasks (book flipping and belt assembly) while outperforming both retraining-frow waaeiiig-ffoo--ratttteo honmseraathh and finetuning approaches. Through extensive real-world experiments, we provide practical guidance for implementing effective DAgger in real-world robot learning tasks. Result videos are available https: /ttp///cpppllo/c///////// wemoompllantt iintt-i-------- resi dual dagger g hithhu..... io/
On-Policy Delta
Correction Data
Base Policy
Whhler Motion Force
Residual
Corrections
Pollcy
(a) Correction Data Collection
(b) Policy Update
(c) Deployment: Base Residual Policy
Figure 1: CReDAgger. To improve rohot manipulation policy, propose compliant intervention interface (a) for collecting human correction data, and this data update compliant residual policy (b), and thoroughly study their effects by deploying updated policy on two contact-rieh manipulation tasks in real world (c).
Introduction
Learning from human demonstrations has seen many recent successes in real-world robotic tasks [9, 21, 41, 48, 42]. However, to obtain a successful policy, human demonstrators often have to repeatedly deploy a policy and observe its failure cases, then collect more data to update the policy until
"Equal Contributions
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

succeeds. This process is broadly referred to as Dataset Aggregation (DAgger) [36, 23]. However, doing DAgger effectiivey for real-world robotic problems still faces the following challenges:
How to collect informative human correction data? DAgger is most effective when the correction data is within the original policy's induced state-action distribution [36]. In practice, the common approach is either (I) collecting offline demonstrations that cover the policy's ' typical failure sce- narios [8], or (2) human taking over robot control during policy deployment [37, 32]. However, in both cases, the human demonstrator has no access to the original policy's 's behavior and may deviate excessively from it. Human taking over additionally introduces force discontinuity when they do not instantly reproduce the exact same robot force. This is partially due to the lack of effective correction interfaces that support precise and instantaneous intervention.
How to effectively update the policy with new data? Prior methods for improving a pretrained policy with additional data include (1) retraining the policy from scratch with the aggregated dataset [23], which can be computationally expensive; (2) finetuning the policy with only the additional data [41, 17, 7J, which is sensitive to the quality of the new data [51], and (3) training a residual policy separately on top of the pretrained policy, which is typically done with Reinforcement Learning [2, 51J or Imitation Learning [5], both require a large number of samples.
In this work, we address these questions by proposing an improved system Compliant Residual DAgger (CR-DAgger) consisting of two critical components:
Compliant Intervention Interface. We propose an on-policy correction system based on kines- thetic teaching to collect delta action without interrupting the current robot policy. Leveraging compliance control, the interface lets humans directly apply force to the robot and feel the mag- nitude of their instantaneous correction. Unlike take-over corrections, our design allows smooth transition between correction/no correction mode, while providing direct control of correction magnitudes.
Compliant Residual Policy. Leveraging the force feedback from our Compliant Intervention Interface, we propose a residual policy formulation that takes in an extra force modality and predicts both residual motions and target farget which can fully describe the human correction behavior. The Compliant Residual Policy is force-aware, even when the base policy is position- only. We show that our residual policy formulation learns effective correction strategies using the data collected from our Compliant Intervention Interface.
Together, our system significantly improves the success rate of precise contact-rich robot manipulation tasks using small amount of additional data. We demonstrate the efficacy of our method on two challenging tasks involving long horizons and sequences of contacts: book flipping and belt assembly. We improve over the base policy success rate by over 60%, while also outperforming retrain-from- scratch and finetuning under the same data budgets. In summary, our contributions are:
A Compliant Intervention Interface, a system that allows humans to provide accurate, gentle, and smooth corrections in both position and force to a running robot policy without interrupting it. A Compliant Residual Policy, policy formulation that seamlessly integrates additional force modality inputs and predicts residual motions and forces.
A practical guide for efficient DAgger based on extensive world studies for critical but often overlooked design choices, such as batch size and sampling strategy. Our hardware design, training data, and policy code can be found here.
2 Related Work
-uuman-n-the-Loop Corrections for Robot Policy Learning. The original DAgger work [36] requires the demonstrator to directly label actions generated by the policy. In robotics, practical variation is to let the human take over the robot control and provide correct action directly [23]. Such human correction motion can be recorded with spacemouse [7, 29], joystick [37], smartphone [32], or arm-based teleoperation system [18, 19, 41]. We instead proposes a novel kinesthetic teaching system with compliance controller that allows the demonstrator to apply delta corrections while the robot policy is still running, and additionally records force feedback. Our results show that both the delta correction data and the force data are crucial to the success of the learned policy.
Improving Pretrained Robot Policies with New Data. The most direct approach to improving a pretrained policy with new correction data is to retrain the policy the aggregated dataset, combining

---

Take-Over Correction
(prior works)
Camera
force discontinuity
Button
Handle
Delta Correction
(ours)
Manipulator
Force Sensor
Humain Conrection Robot Execution
Figure 2: Compliant Intervention Interface characterized by kinesthetic correction hardware setup where humans hold on the handle and apply forces to correct robot execution, providing on-policy delta corrections.
prior demonstrations with new feedback [32, 37]. Alternatively, Reinforcement Learning (RL) offers a framework to incorporate both offline and online data, either by warm warm- starting replay buffers [31, 3] or by using offline data to guide online Lnn-ttnnng [49, 43]. When policies are trained on large-scate human demonstration datasets [6, 38, 25, 34, 24, 44, 45J, retraining becomes impractical, especially when the original data inaccessible, In such cases, fine-tuning with only the new data is a common solution, using either imitation learning [29, 17, 41] or RL [33, 7J. Another line of work introduces an additional residual model on top of the original policy. These residual policies can be trained with RL in simulation [51, 2, 15], but suffers from sim-to-real challenges. Training residual policy in the real world usually requires a large number of samples [S, 22], intermediate scene representation [14], consistent visual observations between training and testing [13, 16], making the approach hard to adopt in practice. In this work, we introduce a practical data collection system and an efficient residual policy learning algorithm for oong-horizon, contact-rich manipulation tasks. Our approach requires only a small amount of real-world correction data and supports integration of additional sensory modalities present in the original model, leading improved policy performance.
3 gR-DAgger Method
Our goal is to improve a pretrained robot policy with a small amount of human correction data. To achieve this, we propose a Compliant Intervention Interface (8 3.1) that enables precise and intuitive on-policy human correction data collection, and Compliant Residual Policy ($ 3.2) that efficiently learns the correction behaviors to be used on top of the pretrained policy. Throughout the paper, we use the term base policy to refer to the pretrained policy without online improvements.
3.1 Compliant Intervention Interface
Correction data is collected by human demonstrators to rectify policy failures. Unlike initial demon- strations that establish baseline behaviors, correction data specifically targets failure modes observed during policy deployment. Correction data is most effective when it corrects failures in policy-induced state distributions [36]. The interface through which these corrections are collected significantly impacts the quality of correction data, which should be intuitive for demonstrators, capture critical corrective information at precise moments of failures, and facilitates collecting data close to the base policy state-action distribution.
There are two types of correction collection methods: Off-policy correction is when humans observe failures of the base policy during deployment, and then recollect extra offline demonstrations to address failure cases. This approach is most commonly used for improving Behavior Cloning policy performance due to its simplicity it requires additional infrastructure beyond the original data collection setup. However, the resulting demonstrations may fail to cover all the failure cases or deviate from the original state-action distribution. We focus on on-policy correction instead, where humans can monitor policy execution and intervene on the spot when failures occur or are anticipated. This approach allows humans provide corrections more targeted to the base policy's failure cases. However, challenges still exist for an intervention system:
Non-smooth transitions. Intervention in robotics is typically implemented by lake-over correction: letting human take complete control and overwrite robot policy. As the underlying control abruptly switches between robot policy and human intention, disturbances are introduced due to policy
3

---

Retrain Finetune (Prior Work)
Sinit
Compliant Residual Policy (Ours)
D mait
correct
A poses (0.1c)
AP
Its Pa img. proprio
π base
A7
Retrain
Finetune
D Doorrect
AAA
1, Pt-
T base
AI
delta poses (0.02s)
Admittance Controller
pes
Robot
F
A target (0.02s)
Figure 3: Policy Update Methods. Left: Common policy update methods retraining and finctuning. Right: Ours. The base policy runs at 1 Hz. It takes in images 4 and proprioceptions P and predicts 32 frames of end-effector poses Af A,.,A, spaced 0.1 Seconds apart. The Compliant Residual Policy runs at 50 Hz. It takes in additional force inputs F; and predicts 5 frames of delta poses AA," (AAg,,Ae Mi......AA, 1z, and target
forces A/ spaced 0.02 Seconds apart. The combined poses of A,, and AAf,, and target forces A/ are taken by an admittance controller to command the robot.
inference and human response latency, especially when the robot is withholding external forces. The recorded data thus may include undesired actions that do reflect the human's intention.
Distribution shift. The human-intervenet state-action distribution may deviate significantly from the original distribution. Additionally, the non-smooth transition above could bring disturbances and add to the distribution shift.
Indirect correction brings errors. Correction is commonly implemented via teleoperation interfaces such as spacemouse or joysticks [17, 7]. With spatial mismatch and teleoperation latency, it is hard for the demonstrator to instantly provide accurate corrections upon intervention starts without going through a short adjustment period.
Missing information. The recorded correction data need to fully describe the human's intended action. Simply recording the robot's position is not sufficient, since it may be under the influence of human correction force and will cause different result when testing without human.
We propose a Compliant Intervention Interface with the following designs to solve those challenges:
Delta correction instead of take-over correction. Unlike take-over correction, where the demon- strator has no idea of the policy's 'ss original intention once taking over, we propose novel on-policy delta correction method: we let the robot policy executes continuously while the human applies forces to the robot with a handle mounted on the end effector, resulting in delta actions top of the policy action. The human demonstrator can always sense the policy's s intention through haptic feedback, and easily control the magnitude of intervention by the amount of force applied to the handle. As a result, delta correction ensures smooth intervention data and limits the human from providing very large corrections. The approach is also intuitive as the human directly move the robot towards desired correction directions.
Correction interface with compliance control. In order to apply delta correction over a running policy, we provide compliant interface that allows humans to safely intervene and apply force to the robot to affect its behaviors at any time, as shown in Fig. 2. We design kinesthetic correction hardware setup with a detachable handle for human to hold when correcting, and allows easy twop-swapping for different tasks. We run a compliance controller (specifically admittance control) in the background to respond to both contact forces and human correction forces, allowing the human to influence but not completely override the policy execution. The admittance controller uses a constant stiffness ~ 1000 N/ to allow easy human intervention and ensure accurate tracking.
recording with buttons and force sensor. Our interface additionally includes an ATI 6-D force sensor to directly measure contact forces, and a single-key keyboard placed on the handle to record the exact timings of correction starts/ends. Both the policy's 's original commands and the human's 's delta corrections are recorded, along with force sensor readings during the interaction.
Correction
3.2 Compliant Residual Policy
Given the correction data, there are multiple ways to update the policy. Common practices include retraining the base policy from scratch with both initial data and correction data, and finetuming the

---

base policy with only the correction data. However, retraining is costly as it requires updating the entire base policy network from scratch with all the available data. It also requires access to the base policy's s initial training data, which might not be accessible for many open source pretrained models. The amount of correction data is significantly smaller than the initial training data, thus simply mixing them together makes the policy hard to gain effective corrective behaviors. While finetuning allows updating partial policy network parameters with new data only, its training stability can be easily affected by the distribution mismatch between the correction data and initial training data. Moreover, both retraining and finetuning can only update the policy with its fixed network architecture while being unable to incorporate new inputs and outputs. We propose compliant residual policy trained only on the correction data to refine base policy's 's position actions and predict additional force actions.
Compliant residual policy formulation. Our policy directly learns corrective behavior from the human delta correction data, as shown Fig. 3. It takes as input the same visual and proprioceptive feedback as the base policy but with shorter horizon. It also takes in an extra force modality, which available using our Compliant Intervention Interface. The policy outputs five frames of actions at a time, corresponding to 0.1 s of execution time when running at 50 Hz. The action space is I 15-dimensional: the first nine dimensions represent the SE3 delta pose from the base policy action to the robot pose command [8], while the later six dimensions represent the expected wrench (force and torque) the robot should feel from external contacts. Both the robot pose command and the expected wrench are sent to standard admittance controller for exccution with compliance.
The residual policy directly uses the base policy's ' frozen image encoder [35, 1, 46] to extract an image embedding, temporal convolution network [39] to encode the force vectors, followed by fully-connected layers to decode actions.
Ad Advanaages This formulation provides the following advantages:
Sample-efficien learning. The residual policy's ss network is light-weight (~2MB trainable weights) and only requires a small amount of correction data to train (50~100 demonstrations), Incorporating new sensor modality. Compared to retraining and finetuning methods that are limited to the base policy's 's network architecture, residual policy can incorporate new sensor modality. This allows taking any position-based pretrained policy and turning it force-aware simply by collecting a small amount of correction data with force modality.
High-frrquency inference. The tightwweight residual policy runs at higher frequency than the base policy, incorporating high-frequenee force feedback and enabling reactive corrective behaviors. This reactivity is particular important for error correction during contact events.
Training strategy. In prior work, a residual policy is trained either in simulation with RL [2, 51] to give it sufficient coverage of the state distribution, or in the real world with pre-collected behavior cloning data [31]. In this work, we train the Compliant Residual Policy completely on the small amount of new real-world correction data with the following strategies:
Ensure sufficient coverage of in-distribution data. Human correction tends to be frequent around a few key moments of the task. A residual trained correction data alone can extrapolate badly around states where no correction is provided. To help the residual policy understand when not to provide corrections, we: (1) include the correction data for training but label it as zero residual actions; (2) collect a few trajectories where the demonstrator always holds the handle and marks the whole trajectory as correction even when the correction is small or zero. Details are in $ A.3.
Prioritize correction data over co-correction (zero residual action) data. Similar to [29], we alter the data sample frequency during training based on whether they have human correction or not. Specifically, since the moment of correction start indicates where the current policy performs badly followed by immediate action to fix it, we sample data more frequently for a short period immediately after correction starts. Our real- world ablations ($ 4.5) demonstrate that our training strategies improve the quality of the residual policy and the overall success rate.
Evaluation
For each task, we train diffusion policy [8] with 150~400 demonstrations as the base policy. We first deploy the base policy and observe its performance and failure modes. Next, from the same base policy, collect 50~ 100 correction episodes with each data collection method. Then, we update the policy using each network updating method and training procedure. Finally, we deploy the updated policies and evaluate their performance under the same test cases. Details of tasks and comparisons are described below.
5

---

(a)
Base Compliant Residual Policy Rollout
Start
Insert
Rotate
Push
Retract
(b) Test Configs Far from edge
(c) Base Policy Failure Cases
Train
Close to edge
Insert too high
Retract too early
Figure 4: Book Flipping Task. (a) Policy rollout [Compliant Residuall policy trained with [On-Policy Delta] data, demonstrating accurate insertion motions and forceful pushing strategy. (b) Different test scenarios. (c)
Typical failure cases of the base policy: inserting too high above the book and missing the gap; retracting the fingers before the books steadily stand.
(a) Base Compliant Residual Policy Rollout
Thread pulley
Thread 2nd pulley
over pulley
Release
(b) Test Configs
(c) Base Policy Failure Cases
Missed the slot
Slipped off the pulley
ball height
Figure 5: Belt Assembly Task. Policy rollout [Compliant Residuall policy trained with [On-Policy Delta] data, demonstrating accurate force-position coordination and adaptation. (b) Different test scenarios. (c) Typical failure cases of the base policy: missing the slot by going too high above the pulley; tilting the belt and causing it to slip off the pulley.
4.1 Contact-Riae Manipulation Tasks
Book Flipping: As shown in Fig. 4 (a), this task is flip up books on a shelf. Starting with one or more books lying flat on the shelf, the robot first insert fingers below the book, then rotate the book up and push them firmly against the shelf to let them stand on their own. The base policy trained with 150 demonstrations.
This task is challenging it involves rich of physical contacts and forceful strategies [20]. A position-only strategy always fails immediately by triggering large forces, so we execute all policies through the same admittance controller. The task success requires high precision in both motion and force to accurately align the fingers with the gap upon insertion, and to provide enough force to rotate heavy books and make the books stand firmly.
Each evaluation includes 20 rollouts on 4 test cases (5 rollouts each), as shown Fig. 4 (b): I) flipping single book (three seen and two unseen books), initially far from the shelf edge; 2) flipping a single book close to the shelf edge; 3) flipping two books together (combinations of three seen and three unseen books), initially far from the shelf edge; 4) flipping two books close to the shelf edge. We use the same initial configurations for all evaluations.
Belt Assembly: As shown in Fig. 5 (a), this task is to assemble thin belt onto two pulleys, which is part of the NIST board assembly challenge [26]. Starting with the belt grasped by the gripper, the robot needs to first thread the belt over the small pulley, next move down while stretching the belt to

---

thread its other side on the big pulley, then rotate 180 around the big pulley to tighten the belt, and finally pull up to release the belt from the gripper. The base policy trained with 405 demonstrations.
The task is challenging as it requires both position and force accuracy throughout the process. Specifically, the belt is thin and soft so the initial alignments onto the pulleys visually ambiguous. Also, since the belt is not stretchable, there is more resistance force and less position tolerance as the belt approaches the second pulley, requiring policy with good force-position coordination and adaptation. We ran 32 rollouts across four different initial board positions and four grasp locations (Fig. (b)).
4.2
Base Policy and its Failure Modes
We trained a diffusion policy [8] that takes in past images from wrist-mounter camera and robot proprioception observations, and predicts a future position-bas based action trajectory. To isolate the contribution of force inputs versus human corrections, we trained diffusion policies with and without force inputs as baselines for the belt assembly task.
The book flipping base policy achieves a 40% success rate with the following common failure cases (Fig. 4 (c)): (I) Missed insertion. The fingers initially go too high above the book or aims for the gap between the two books, failing to properly insert beneath the books. (2) Incomplete flipping. At the last stage, the policy retracts the blade before the book can stand stably, causing it to fall back.
The belt assembly base policy achieves 15.6% success rate. Adding force input increases the base policy success rate to 43.8%. Common failure cases include (Fig. 5 (c)): (1) Missed slotting: the fingertip goes too high or too low, causing the belt to miss the slot on the big pulley, (2) Belt slippage: the fingers pull the belt in the wrong direction, causing the belt to tilt and slip off the pulley.
4.3 Comparisons
We compare CR-DAgger with baselines across two dimensions: correction method and policy update method. We present the quantitative results in Fig. 6, and explain key findings in 4.4.
Correction data collection methods. We compare our Compliant Intervention Interface with the two most commonly used correction data collection strategies:
Observe-then-Cone Coolect includes two steps: first, the policy is deployed and human demonstrators observe the initial settings that could cause failures; then, demonstrators provide completely new demonstrations starting from similar initial settings. As explained in $ 3.1, this type of offline correction potentially misses critical timing information, and the resulting demonstrations may deviate from the policy's original behavior distribution.
-ovee-Corent (HG-DAgger) [23] is another common correction strategy where human demonstrators monitor policy execution and take complete control when failures are anticipated. We implement Make-over-Correctio
on our Compliant Intervention Interface by cleaning up command buffer to the compliance controller and switching stiffness zero upon correction starts, so the robot policy does not affect the robot during correction. However, as explained in $ 3.1, take-over correction introduces an abrupt transition around control authority switching, which may cause distributional discontinuities in the training data.
On-Policy Delta (Ours): the details are described in $ 3.1.
Policy update methods. We compare with two common policy update methods:
Retrain Policy: Retrain the base policy using both the original training data and the correction data from scratch. As explained in $ 3.2, this approach is reliable but may require access to the orignal data and large amount of new data to work well.
Finetune Policy: Finetune the base policy using only the correction data (freezing visual encoders). As explained in $ 3.2, this approach can be sensitive data quality and distribution shifts.
Fintune Policy with KL Regularization: A recent method [11] that stabilizes finetuning training by encouraging the predicted action to be close to the training data distribution.
Residual Policy: an ablation our method where force is removed from both input and outputs.
Compliant Residual Policy (Ours): Residual policy update with additional force input and outputs, see details in 3.2.
7

---

Book Plipping
Belt Assembly
Obneeve Cher-Calect
14-
Sucoes
20-3
28
28
wie Forne
Resral
Flavtave
Bestdiaa
ok (oung)
Nove ere Ferce
Bese Ferce
Racrmm
Bestdoal
ak (Ours)
Figure 6: Results. We compare CR- -Rgger across two dimensions: correction method and policy update method. The result shows that our [Compliant Residual (CR)) policy trained with (On-Policy Deltal data is able to improve upon base policies on both tasks and outperforms other variations.
4.4 Key Findings
Finding 1: Compliant Residual Policy is able to improve base policy by large margin. As shown in Fig. 6, [Compliant Residual] policy trained with [On-Policy Delta] data improves the base policy success rate by 60% and 50% on the two tasks respectively. It effectively learns useful corrective strategies from the limited demonstrations. For example, in the book flipping task, the policy learns to pitch the fingers down more before finger insertion to increase the insertion success; in the belt assembly task, the policy learns to correct the height of the belt when misaligning to the pulley slot. Results are best viewed in our supplementary video.
Finding 2: Residual policy allows additional useful modality to be added during correc- tion. [Compliant Residual] policy performs significantly better than other methods without force (45% higher success rate than the best position-only baseline on the book task and 53% higher on the belt task) it can both take in force feedback that indicates critical task information and predict adequate contact forces to apply. For example, the last stage of the book flipping task requires the robot to firmly push the book against the shelf wall to let stand on its own. [Compliant Residual] policy predicts large pushing forces at this stage to make the books stand stably with a 100% success rate, while [Residual]'s success rate drops from 70% to 35% ($ A.2). The second stage of the belt assembly task (threading the belt on the large pulley) requires delicate belt height adjustments under ambiguous visual information due to occlusions and the lack of depth. [Compliant Residual) policy learns to move along the pulley to find the slot when the finger touches the top of the pulley.
Finding 3: Smooth On-Policy Delta data enables stable residual policy. [Compliant Residual] policy has 45% higher success rate when trained on [On-Policy Delta] data instead of [Take-over- Correction] data on the book flipping task. Residual policy trained with (Take-over-Cort) Correction] data sometimes exhibits large noisy motions that trigger task failures, such as retracting the fingers too early in the book flipping task. On the contrary, residual policy trained with [On- [On-policy Delta] data have much smoother action trajectories and better reflect human's 's correction intentions, providing suitable magnitudes of corrections.
Finding 4: Retraining base policy is stable but learns correction behavior slowly. Retraining from scratch with the initial and correction data together leads to policies with stable motions. However, its behavior is less affected by the small amount of correction data compared to the dominant portion of initial data, leading to insignificant improvements over the base policy (1.67% success rate drop on the book task averaged across all data collection methods, 18.8% success rate improve on the belt task, both are much less improvements than [Compliant Residual]).
Finding 5: Finetuning base policy is unstable. Policy finetuning with either correction data has the worst performance across all policy update methods and even underperforms the base policy (30% success rate drop on the book task averaged across all data collection methods, 15.6% drop on the belt task). The finetuned policy predicts unstable and noisy motions, quickly leading to -uitt--distributionn states, such as inserting too high in the book flipping task and drifting away from the board in the belt assembly task. This is likely due the distribution mismatch between the base policy training data and correction data, causing training instabilities. Adding KL-regularization effectively reduced the noisy behavior, however, the overall success rate is still lower than other baselines.
4.5 Ablations
We study two important design decisions with ablation studies on the book flipping task.
8

---

Training frequency and batch size. One important parameter in DAgger is the batch size be- tween policy updates. With a smaller batch size, the policy is updated more frequently, then new correction data can better reflect the most recent policy behavior. However, DAgger with small batch sizes known to suffer from catastrophic forgetting [27, 12] since it finetunes neural networks on data with non-stationary distribution. Common solutions include retrain- ing the residual policy at the end of DAgger using all available correction data collected from all the intermediate residual policies [41]. Another way is to rely the base policy training data as a normalizer [17]. In this work, we empirically found that larger batch sizes can ef- fectively stabilize residual training. With batch size 50, the book flipping task reach 100% with one batch, while the belt assembly task performs better with two batches. We compare method with smaller batch size the book flipping task, where warm up the residual with 20 episodes of initial correction data, then update every ten more episodes for three times.
Requrecy
Finding: Large-batch DAgger more suitable for training Compliant Residual Policy. The small-batch training becomes unstable and the demonstrator needs to provide large magnitudes of corrections as the number of iterations in- creases. During evaluation, the final policy al- ways fails by inserting too high, while our single- batch policy achieves 100% success rate with the same amount of data and training epochs.
75
30
batch 9be=19
b00ch 926-55
Unifomm
Dense Anonnd Dense After
Figure 7: Effect of Training Frequency and Sample. Batch size=50 leads to more stable training and dense sampling after correction starts achieves better perfor- mance on the book flipping task.
Sampling strategy during training. The start of a human intervention contains critical infor- mation of the timing and direction of correction. Accurate delta action predictions right after correction starts are important for reactive corrective behaviors. We investigate three strategies for sampling from online correction data during training: 1. Uniform sample, where the whole episode is sampled uniformly. 2. Denser sample around the start of a human intervention, and 3. denser sample only after the human intervention starts. For 2 and 3, uniformly increase the sample frequency four times for a fixed period before and/or after intervention starts.
Finding: Sampling denser right after intervention starts leads to more reactive and accurate cor- rections. As shown in Fig. 7 (right), the best performance comes from densely sampling after the beginning of interventions. Sampling denser around the start of human intervention also adds more samples right before the intervention starts, which where humans observe signs of failures. These are mostly negative data, and using them for training decreases the policy success rate.
5 Conclusion and Discussion
In this work, we evaluate practical design choices for DAgger in real-world robot Icarning, and provide system, CR- CR-DAgger, to effectively collect human correction data with a Compliant Intervention Interface and improve the base policy with a Compliant Residual Policy. We demonstrate the effectiveness of our designs by comparing them with variety of alternatives on four contact-rich manipulation tasks.
Limitations and Future Work.
The base policy should have reasonable success rate for the residual policy to learn effectively. From our experiments, we recommend starting to collect correction data for the residual policy when the base policy has least 10% ~ 20% success rate. future direction to derive theoretical guidelines for the trade-off between the base and residual improvements.
Throughout this work, we use MLP as the action head of our Compliant Residual Policy and directly regress the actions. Although it works well in our tasks, may experience difficulty for tasks that involve more distinctive action multi-modalities. More expressive policy formulations, such as Flow Matching [28, 6] might be useful for these tasks.
Our data collection system is based on kinesthetic teaching. Although it provides richer data with higher quality than teleoperation as explained in the paper, it may require more labor during training data collection since the demonstrator needs to grasp the handle on the robot.

---

Acknowledgment
We would like thank Chendong Xin for his support on experiments and rebuttal. We would like to also thank Eric Cousineau, Huy Ha, and Benjamin Burchfiel for thoughtful discussions on the proposed method, thank Mandi Zhao, Maximillian Du Calvin Luo, Mengda Xu, and all REALab members for their suggestions on the experiment setup and the manuscript. This work was supported part by the NSF Award #2143601, #2037101, and #2132519, 19, Sloan Fellowship, Toyota Research Institute, and Samsung. We would like to thank Google and TRI for the URS robot hardware. The views and conclusions contained herein are those of the authors and should not be interpreted as necessarily representing the official policies, either expressed implied, of the sponsors.
10

---

References
[1] Shir Amir, Yossi Gandelsman, Shai Bagon, and Tali Dekel. Deep vit features as dense visual descriptors. arXiv preprint arXiv:2112.05814, 2(3):4, 2021.
[2] Lars Ankile, Anthony Simeonov, Idan Shenfeld, Marcel Torne, and Pulkit Agrawal. From imitation refinement-rrsiduull rl for precise assembly. arXiv preprint arXiv:2407.16677, 2024.
[3] Philip Ball, Laura Smith, Nya Kostrikov, and Sergey Levine. Efficient online reinforcement learning with offline data. In International Conference on Machine Learning, pages 1577-1594. PMLR, 2023.
[4] Maria Bauza, Antonia Bronars, Yifan Hou, Ian Taylor, Nikhil Chavan-Dafle, and Alberto Rodriguez. Simple, visuotactile method learned in simulation to precisely pick, localize, regrasp, and place objects. Science Robotics, 9(91 readi888888 2024.
[5] Homanga Bharadhwaj, Roozbeh Mottaghi, Abhinav Gupta, and Shubham Tulsiani. Track2act: Predicting point tracks from internet videos enables generalizable robot manipulation. In European Conference on Computer Vision, pages 306-324. Springer, 2024.
[6] Kevin Black, Noah Brown, Danny Driess, Adnan Esmail, Michael Equi, Chelsea Finn, Niccolo Fusai, Lachy Groom, Karol Hausman, Brian Ichter, et To: A aaiiin-lnnguag-acttion flow model for general robot control. arXiv preprint 40X 164, 2024.
[7] Yuhui Chen, Shuai Tian, Shugao Liu, Yingting Zhou, Haoran Li, and Dongbin Zhao. Con- rft A reinforced fine-tuning method for vla models via consistency policy. arXiv preprint arXiv:2502.05450. 2025.
[8] Cheng Chi, Zhenjia Xu, Siyuan Feng, Eric Cousineau, Yilun Du, Benjamin Burchfiel, Russ Tedrake, and Shuran Song. Diffusion policy: Visuomotor policy learning via action diffusion. The International Journal of Robotics Research, page 02783649241227666. 2023.
[9] Cheng Chi, Zhenjia Xu, Chuer Pan, Eric Cousincau, Benjamin Burchfiel, Siyuan Feng, Russ Tedrake, and Shuran Song. Universal manipulation interface: to-the-wild robot teaching without in-the-wild robots. arXiv preprint arXiv:2402. 10329, 2024.
[10] Hojung Choi, Jun En Low, Tae Myung Huh, Gabriela A Uribe, Seongheon Hong, Kenneth AW Hoffman, Julia Di, Tony Chen, Andrew A Stanley, and Mark R Cutkosky. Coinft: A coin-sized, capacitive 6-axis force torque sensor for robotic applications. arXiv preprint 1922555 2025.
[11] Ying Fan, Olivia Watkins, Yuqing Du, Hao Liu, Moonkyung Ryu, Craig Boutilier, Picter Abbeel, Mohammad Ghavamzadeh, Kangwook Lee, and Kimin Lee. Dpok: Reinforcement learning for fine-tuning texl-to-image diffusion models. Advances in Neural Information Processing Systems, 36:79858-798855 2023.
[12] Ian J Goodfellow, Mehdi Mirza, Da Xiao, Aaron Courville, and Yoshua Bengio. An empirical investigation of catastrophic forgetting in gradient-base based neural networks. arXiv preprint 132.61131..6211, 2013.
[13] Irmak Guzey, Yinlong Dai, Ben Evans, Soumith Chintala, and Lerrel Pinto. See to touch: Learning tactile dexterity through visual incentives. In 2024 IEEE International Conference on Robotics and Automation (ICRA), pages 13825-13832. IEEE, 2024.
[14] Irmak Guzey, Yinlong Dai, Georgy Savva, Raunaq Bhirangi, and Lerrel Pinto. Bridging the human to robot dexterity gap through object-oriented rewards. arXiv preprint arXiv:2410.23289, 2024.
[15] Siddhant Haldar, Vaibhav Mathur, Denis Yarats, and Lerrel Pinto. Watch and match: Super- charging imitation with regularized optimal transport. In Conference on Robot Learning, pages 32-43. PMLR, 2023.

---

[16] Siddhant Haldar, Jyothish Pari, Anant Rai, and Lerrel Pinto. Teach robot to fish: Versatile imitation from one minute of demonstrations. arXiv preprint 2rXiv:2303.01497, 2023.
[17] Zhanpeng He, Yifeng Cao, and Matei Ciocarlie. Uncertainty comes for free: Human-in-ther -oop policies with diffusion models. arXiv preprint arXiv:2503.01876 2025.
[18) Ryan Hoque, Ashwin Balakrishna, Ellen Novoseller, Albert Wilcox, Daniel S Brown, and Ken Goldberg. Thriftydagger: Budget-aware novelty and risk gating for interactive imitation learning. arXiv preprint arXiv:2109.0880 08273, 2021.
[19] Ryan Hoque, Ashwin Balakrishna, Carl Putterman, Michael Luo, Daniel S Brown, Daniel Seita, Brijen Thananjeyan, Ellen Novoseller, and Ken Goldberg. Lazydagger: Reducing context switching in interactive imitation learning. In 2021 IEEE 17th international conference on automation science and engineering (case), pages 502-509. IEEE, 2021.
[20] Yifan Hou, Zhenzhong Jia, and Matthew Mason. Manipulation with shared grasping. arXiv preprint arXiv:2006. 200,009000002999 2020.
[21] Yifan Hou, Zeyi Liu, Cheng Chi, Eric Cousineau, Naveen Kuppuswamy, Siyuan Feng, Benjamin Burchfiel, and Shuran Song. Adaptive compliance policy: Learning approximate compliance for diffusion guided control. In 2025 IEEE International Conference Robotics and Automation (ICRA), pages 4829-4836, 2025. doi: 10.1 N0.110///CRA55545..00000 1112852..
[22] Tobias Johammink, Shikhar Bahl, Ashvin Nair, Jianlan Luo, Avinash Kumar, Matthias Loskyll, Juan Aparicio Ojea, Eugen Solowjow, and Sergey Levine. Residual reinforcement learning for robot control. In 2019 international conference on robotics and automation (ICRA), pages 6023 6029. IEEE, 2019.
[23] Michael Kelly, Chelsea Sidrane, Katherine
and Mykel J Kochenderfer. Hg- dagger: Interactive imitation learning with human experts. In 2019 International Conference on Robotics and Automation (ICRA), pages 8077-8083 IEEE, 2019.
[24] Alexander Khazatsky, Karl Pertsch, Suraj Nair, Ashwin Balakrishna, Sudeep Dasari, Siddharth Karamcheti, Soroush Nasiriany, Mohan Kumar Srirama, Lawrence Yunliang Chen, Kirsty Ellis, et al. Droid: large-scale in-the-wild robot manipulation dataset. arXiv preprint arXiv:2403. 12945, 2024.
[25] Moo Jin Kim, Karl Pertsch, Siddharth Karamcheti, Ted Xiao, Ashwin Balakrishna, Suraj Nair, Rafael Rafailov, Ethan Foster, Grace Lam, Pannag Sanketi, al. Openvla: An open-source vision-language-action model. arXiv preprint arXiv:2406.09240 2024.
[26] Kenneth Kimble, Karl Van Wyk, Joe Falco, Elena Messina, Yu Sun, Mizuho Shibata, Wataru Uemura, and Yasuyoshi Yokokohji. Benchmarking protocols for evaluating small parts robotic assembly systems. IEEE robotics and automation letters, 883-888,,,, 2020.
[27] James Kirkpatrick, Razvan Pascanu, Neil Rabinowitz, Joel Veness, Guillaume Desjardins, Andrei A Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agmieszka BarbbskaBarrinska,, et al. Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences, 1 (14(1)::3521-3555 2017.
[28] Yaron Lipman, Ricky TQ Chen, Heli Ben-Hamu, Maximilian Nickel, and Matt Le. Flow matching for generative modeling. arXiv preprint arXiv:2210.02744, 2022.
[29] Huihan Liu, Soroush Nasiriany, Lance Zhang, Zhiyao Bao, and Yuke Zhu. Robot learning on the job: Human-in-the-loge -o-p--oopp autonomy and learning during deployment. The International Journal of Robotics Research, page 02783649241273901. 2022.
[30] Yun Liu, Xiaomeng Xu, Weihang Chen, Haocheng Yuan, He Wang, Jing Xu, Rui Chen, and Li Yi. Enhancing generalizable 6d pose tracking of an in-hand object with tactile sensing. IEEE Robotics and Automation Letters, 9(2: 106--113, 2023.
[31] Jianlan Luo, Charles Xu, Jeffrey Wu, and Sergey Levine. Precise and dexterous robotic manipulation via human-in-the-le -ooo---oop reinforcement learning. arXiv preprint 11..::2410.2/845. 2024.
12

---

[32] Ajay Mandlekar, Danfei Xu, Roberto Savarese.
Yuke Zhu, Li Fei-Fei, and Silvio the-loop imitation learning using remote teleoperation. arXiv preprint arXiv:20/2.06733 2020.
[33] Max Sobol Mark, Tian Gao, Georgia Gabriela Sampaio, Mohan Kumar Srirama, Archit Sharma, Chelsea Finn, and Aviral Kumar. Policy agnostic rl: Offline and online fine-tuning of any class and backbone. arXiv preprint 24Kiv:24/2206688. 2024.
[34] Abby O'Neill, Abdul Rehman, Abhiram Maddukuri, Abhishek Gupta, Abhishek Padalkar, Abraham Lee, Acorn Pooley, Agrim Gupta, Ajay Mandlekar, Ajinkya Jain, et al. Open x- embodiment: Robotic learning datasets and rt-x models: Open x-embodiment collaboration O. In 2024 IEEE International Conference on Robotics and Automation (ICRA), pages 6892-6903. IEEE, 2024.
[35] Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International conference machine learning, pages 8748-8763. PmLR, 2021.
[36] Stéphane Ross, Geoffrey Gordon, and Drew Bagnell. A reduction of imitation learning and structured prediction to no-regret online learning. In Proceedings of the fourteenth interna- tional conference on artificial intelligence and statistics, pages 627-635. JMLR Workshop and Conference Proceedings, 2011.
[37] Jonathan Spencer, Sanjiban Choudhury, Matthew Barnes, Matthew Schmittle, Mung Chiang, Peter Ramadge, and Siddhartha Srinivasa. Learning from interventions: Human-robot interac- tion both explicit and implicit feedback. In 16th roboties: science and systems, RSS 2020. MIT Press Journals, 2020.
[38] Octo Model Team, Dibya Ghosh, Homer Walke, Karl Pertsch, Kevin Black, Oier Mees, Sudeep Dasari, Joey Hejna, Tobias Kreiman, Charles Xu, et al. Octo: An open-source generalist robot policy. arXiv preprint arXiv:2405.12213 2024.
[39] Aaron Van Den Oord, Sander Dieleman, Heiga Zen, Karen Simonyan, Oriol Vinyals, Alex Graves, Nal Kalchbrenner, Andrew Senior, Koray Kavukeuoglu, al. Wavenet: A generative model for raw audio. arXiv preprint arXiv:1609.034499 12, 2016.
[40] Luigi Villani and Joris De Schutter. Force control. In Springer handbook of robotics, pages 190-220. Springer, 2016.
[41] Philipp Wu, Yide Shentu, Qiayuan Liao, Ding Jin, Menglong Guo, Koushil Sreenath, Xingyu Lin, and Pieter Abbeel. Robocopilot: Human-in- inttun--i--t-tttte -hop interactive imitation learning for robot manipulation. arXiv preprint arXiv:2503.07771, 2025.
[42] Haoyu Xiong, Xiaomeng Xu, Jimmy Wu, Yifan Hou, Jcannette Bohg, and Shuran Song. Vision in action: Learning active perception from human demonstrations. arXiv preprint arXiv:2506. /5666, 2025.
[43] Mengda Xu, Manuela Veloso, and Shuran Song. Aspire: Adaptive skill priors for reinforcement learning. Advances in Neural Information Processing Systems, 35:38600-38613, 2022.
[44] Mengda Xu, Zhenjia Xu, Cheng Chi, Manuela Veloso, and Shuran Song. Xskill: Cross embodiment skill discovery. In Conference on robot learning, pages 3536-3555. PMLR, 2023.
[45] Mengda Xu, Zhenjia Xu, Yinghao Xu, Cheng Chi, Gordon Wetzstein, Manuela Veloso, and Shu- ran Song. Flow as the cross-domain manipulation interface. arXiv preprint 1422:22477.1528. 2024.
[46] Xiaomeng Xu, Yanchao Yang. Kaichun Mo, Boxiao Pan, Yi, and Leonidas Guibas. Jacobinerf: Nerf shaping with mutual information gradients. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 16498-16507, 2023.
[47] Xiaomeng Xu, Huy Ha, and Shuran Song. Dynamics-guided diffusion model for robot manipu- lator design. arXiv preprint 5033, 2024.
13

---

[48] Xiaomeng Xu, Dominik Bauer, and Shuran Song. Robopanoptes: The all-seeing robot with whole-body dexterity. arXiv preprint arXiv:2501.05420, 2025.
[49] Patrick Yin, Tyler Westenbroek, Simran Bagaria, Kevin Huang, Ching-an Cheng, Andrey Kobolov, and Abhishek Gupta. Rapidly adapting policies to the real world via simulation- guided fine-tuning. arXiv preprint arXiv:2502.027000 2025.
[50] Wenzhen Yuan, Siyuan Dong, and Edward H Adelson. Gelsight: High-resolution robot tactile sensors for estimating geometry and force. Sensors, 17(12):772, 2017.
[51] Xiu Yuan, Tongzhou Mu, Stone Tao, Yunhao Fang, Mengke Zhang, and Hao Su. Policy decora- tor: Model-agnostic online refinement for large policy model. arXiv preprint arXiv:2412.13660, 2024.
14

---

A Technical Appendices and Supplementary Material
A.1 Control Implementation Details
A.1.1 Compliance Control
Compliance is motor property that describes how motion responds to force. For example, balloon has high compliance, meaning that they deform significantly under external forces. Industrial robots typically have very low compliance, meaning that their motion tracking has little deviation under external forces.
Compliance control refers to the technique that makes rigid robot behave softly using feedback control. It is standard technique widely adopted in the manufacturing industry. lets robots interact with the environment safely without creating huge forces. With compliance, the robot retreats when experiences external forces, larger force leads to bigger position deviations. When there is no external force, compliance control tracks the target position accurately just like position-based control. Compliance control allows specification of the exact desired compliance profile, typically described by parameters such stiffness, damping and inertia.
Impedance control and admittance control are two methods implement compliance control. The users are free to choose the controller that works with their robot. Robots with good position control accuracy (e.g., most industrial robots) can use admittance control. Robots with low gear ratio (e-g. qquasi-direct-driig
robots) can use impedance control. We use admittance control in this work and open-sourced our implementation at https: /ett://gttttt com/ y fan-faunu hou force_orcccccoo _ecetttotttt rol. A good reference for different compliance control schemes is the "Force Control" chapter of the Handbook of Robotics [40].
A.1.2 Control Architecture
Our software system consists of three independent loops:
1. The base policy loop that runs the diffusion policy. The base policy loop updates at about 1Hz, each time predicts 32 frames of actions corresponding 3.2s of future robot positions. In this work we have not optimized the implementation for computation speed.
2. The residual policy loop that runs at approximately 50Hz, each time predicts five frames of delta actions corresponding to 0.1s of delta positions. The delta actions are added to the corresponding base policy actions based one time, before being sent to the low-level compliance controller for execution. The loop rate is limited by our current implementation and can be improved if needed.
3. The admittance controller loop that runs at exactly 500Hz. This loop implements 6D Cartesian compliance on the robot. takes reference positions and forces from the residual policy. When there is zero reference force and no external force, the admittance controller lets the robot track the reference position precisely. When external force exists, the robot will deviate from the reference position like a spring centered the base policy output position.
Apart from the above controller/policy loops, each hardware (e.g. camera, force-torque sensor) has standalone driver loop maintaining 1. communication with the hardware, and 2. buffers for action and feedback for this hardware.
A.2 Stage-Wise Success Rate
We report the success rate of the book flipping task into three key stages.
A.3
Correction Data Decomposition
As mentioned the "Training strategy" part 3.2, we used two strategies to ensure the residual policy behaves stably around low correction data regions. The first strategy is to include the no correction portion of online data for training and label them with zero residual actions. The second strategy is to collect few trajectories (15 out of the 50 total correction episodes) in which the
15

---

then
On-Policy
Book Flipping: insertion
Book Flipping: Rotation (>80")
Book Flipping: Release & Stands
60
(%)
Base (65%)
Base (65%)]
70
40
15
20
Pinetune
Residual
(Ours)
Retrain
Ainetune
Residual
(Ours)
Retrain
finetune
Residual
(Ours)
Figure 8: Stage-wise Success rafes, Each row represents the results for one task, while cach column shows the success rate up to corresponding stage.
demonstrator marks the whole trajectory as correction, even when the correction is small or zero. In practice, we find that the first strategy works better when the base policy is more stable and has higher success rate, while the second strategy works better otherwise. In our experiments, we use the first strategy for the book flipping task and use the second strategy for the belt assembly task.
A.4 Experiments Compute Resources
We desktop with NVIDIA GeForce RTX 4090 GPU for training and deployment.
A.5 Hardware Design
Our kinesthetic correction hardware setup features a tool interface that allows task-specific tool swapping. For the book flipping task, designed customized fork-shaped tool that easily insert under the books and flip them. For the belt assembly task, we used a standard WSG-50 gripper and fin-ray fingers [9]. An interesting future direction is to leverage generative models for automatic manipulator design [47]. Future work can also incorporate other types of force or tactile sensors, such capacitive F/T sensors [10] and vision-based tactile sensors [50, 30, 4J.
A.6 Broader Impact
CR-DAgger contributes to the field of robotics by improving pretrained real-world manipulation policies with small amount of human correction data. The proposed Compliant Intervention Interface provides an intuitive and safe way for humans to directly interact with robots and correct the robot policy on the spot. We demonstrate significant policy improvements rich manipulation tasks, book flipping and belt assembly, which can lead to useful applications in industry and households.
real-world contact-
16

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: We propose novel system for real-world DAgger, and explain the method in detail ($ 3) and carry out extensive real-world experiments ($ 4).
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification:
We discuss limitations in 5.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
the paper. The authors should use their best
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA]
17

---

Justification:
The paper does not include theoretical results.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless whether the code and data are provided or not)?
Answer: [Yes]
Justification:
We explain our algorithm in detail in 3 and our experiment setups in 4.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how to reproduce that algorithm.
(b) the contribution is primarily a new model architecture, the paper should describe the architccture clearly and fully.
(c) If the contribution is new model (e.g., a large language model), then there should either way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reprodueibility. the case of closed-source models, may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
18

---

Answer: [Yes]
Justification: We submit code and data in supplemental material. Our code base is also open-sourced /tttps::ggittii com/ com/yiffam yifan-houuuu cr- da-ga....
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https: /tappss://iisp/// cc/
public/ /uiidesuisssee/eee CodeSubmi ss sssioooolicy for more details.
While we encourage the release of code and data, we understand that this might be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed reproduce the results. See the NeurIPS code and data submission guidelines (https: for more details.
/nips. cc/ httpssccpppblccccccocc wwwpli/gguiiioooocceooooppppplee
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary understand the results?
Answer: [Yes]
Justification:
The full details are provided with the code.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes]
Justification: We report success rates over 20 to 32 real-world rollouts across various test cases for all evaluated methods ($ 4.4). For robotics experiments involving physical hardware in real-world settings, we chose to report aggregate success rates rather than error bars, as this is the standard practice the field.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).
19

---

The method for calculating the error bars should be explained (closed form formula, call a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
It should be clear whether the error bar is the standard deviation the standard error of the mean.
It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful figures symmetric error bars that would yield results that are out of range (e.g. negative rates).
show in tables or
If error bars are reported in tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification:
We provide details on computer resources in $ A.4.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary failed experiments that didn't 't make it into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https: /neurips......//
Answer: [Yes]
Justification: We have reviewed the NeurIPS Code of Ethics and confirm that the paper conform with it.
Guidelines:
The answer NA means that the authors have reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws or regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive socictal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification:
We discuss the broader impact of the paper in $ A.6.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
20

---

Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), faireess considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification:
The paper poses such risks.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g.. code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: We collect data ourselves and only use open-source (MIT license) codebases and libraries, which are properly cited.
Guidelines:
The answer NA means that the paper does use existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
21

---

If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, papersw swi thoode com/ datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset.
For existing datasets that are re- re-packaged, both the original license and the license of the derived asset (if has changed) should be provided.
If this information not available online, the authors are encouraged to reach out to the asset's s creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification:
We provide documentation along with our code and data.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/moode as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsoureing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification:
The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based the requirements of your country institution) were obtained?
Answer: [NA]
Justification: The paper does not involve crowdsoureing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
22

---

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, formatting purposes and does not impact the core methodology. scientific rigorousness, or originality of the research, declaration is not required.
Question:
Answer: [NA]
Justification: The core method development does not involve LLMs.
Guidelines:
The answer NA means that the method development in this research does not involve LLMs as any important, original, or non-standard components.
Please refer to our LLM policy (https: //nuuriippp cc/ Conferences / 2025/
LLM) for what should should not be described.
23


Reviews:
[{"review_id": "9EJKsnk6lg", "reviewer": "Reviewer_YHb6", "summary": "This paper presents a data-efficient method for improving pretrained policies on contact-rich robot manipulation tasks - by collecting delta action correction data and force feedback from human, and using this data to train a force-aware residual policy. The data collection method employs admittance control for human operators to injecting necessary intervention online while the robot follows base policy actions (10Hz). The collected data is used to train a residual policy that predicts delta action and autoregressively predict expected force (50Hz), which operates on top of the base policy in deployment. \n\nThe paper conducts extensive experiments and studies on two real-world tasks. By comparing between various ablations and baselines, the authors show that the proposed strategy significantly outperform other existing strategies.", "questions": "- How is the frequency of the compliant residual policy to be selected? WIth executed without the residual policy, does the admittance control run at the same frequency as well?", "limitations": "Yes", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 4, "significance": 3, "originality": 3, "strengths_and_weaknesses": "## Strength\n\n- I enjoy reading this paper for its clarity and transparency. The problem it studies is well motivated, and the proposed methods address the problem well, backed by sufficient experimental results.\n- Incorporating force feedback for human correction and policy deployment in contact-rich tasks is a foundamental problem in robot manipulation. I'm glad to see the authors propose a viable solution that effectively improves the success rate.\n- Many of the discussions are informative and insightful. In particular, I like how the method is compared with different correction data collection strategies, as well as different training methods.\n\n## Weakness\n\n- There are two aspects that may explain the performance gain introduced by the complaint residual policy on top of the base policy: one is the human correction, and another is the force modality. By reading the paper, it's not clear to me how these two aspects contribute to the success of complaint residual policy. It would be very helpful if the authors provide more comparison on this, for example, training the base policy with force input as well and compare its results with complaint residual policy.", "was_revised": true, "final_justification": "I will keep my score.", "rebuttal": "> Q1: There are two aspects that may explain the performance gain introduced by the complaint residual policy on top of the base policy: one is the human correction, and another is the force modality. By reading the paper, it's not clear to me how these two aspects contribute to the success of complaint residual policy. It would be very helpful if the authors provide more comparison on this, for example, training the base policy with force input as well and compare its results with complaint residual policy.\n\nWe thank the reviewer for the great question.\nTo isolate and evaluate the contribution of human correction, we added a baseline [Base Policy with Force], where we train a diffusion policy that also takes force input on the same set of demonstration data. As shown in the results below on the belt assembly task, while [Base Policy with Force] improves success rate over [Base Policy] by 28.125%, [Compliant Residual Policy]’s improvement is a much more significant 81.2%, showing the contribution of human corrections in addition to force modality. We’ve also added the rollout videos to the project website.\n\n| Method | Success Rate |\n|--------|--------------|\n| Base Policy | 15.625% |\n| Base Policy with Force | 43.75% |\n| Residual Policy | 28.125% |\n| Compliant Residual Policy | 96.825% |\n\nOn the other hand, our initial submission already shows the contribution of force modality in addition to human correction: the [Residual Policy] baseline is the same as our [Compliant Residual Policy] except that force is removed from both its inputs and outputs. Our [Compliant Residual Policy] consistently outperforms these baselines, demonstrating that force modality is beneficial for learning correction behaviors of contact-rich manipulation tasks, as shown in Sec 4.4 Finding 2. \n\nTherefore, both human correction and force modality contribute to the success of the compliant residual policy - by learning effective correction behaviors, which are often indicated by force modality, the compliant residual policy consistently improves the initial base policy.\n\n\n> Q2: How is the frequency of the compliant residual policy to be selected? WIth executed without the residual policy, does the admittance control run at the same frequency as well?\n\nThe 50Hz frequency of compliant residual policy is upper bounded by the computation time of our entire control loop. We have not optimized our implementation particularly for computational efficiency. This frequency is sufficient for table top manipulation tasks in our experiments.\n\nA separate standard admittance controller is running at low level at 500Hz to make the robot compliant while tracking the target position (and force, if given) from the policy. It runs in its own thread and uses the same parameters across all baselines."}, {"review_id": "ePz48sF2Az", "reviewer": "Reviewer_J472", "summary": "This paper tackles the challenge in collecting correction data for robot policies.  Prior works are mostly based on two approaches: (1) observe-the-collect, an offline method that re-collect successful trajectories after observing failed operation of robots, and (2) take-over-correction, an online method that introduce human's interference when failures are about to occur.  However, these approaches are limited, as the former fails to cover the the policy’s typical failure scenarios and the later introduces non-smooth transitions or errorneous correction due to spatial mismatch and teleoperationlatency.  This paper proposes an online alternative that leverages compliance control to provide smooth and instantaneous action corrections.  Moreover, this paper introduces a compliant residual policy that predicts residual actions and intended force control for the base policy.  Carrying out experiments on real-world manipulation tasks, this paper demonstrations significant performance gain over position-control baselines.", "questions": "1. What is the formulation of the compliance control?\n\n2. What does \"the policy’s original distribution\" at line 107 mean? What does \"distribution shift\" mean at line 116? What does \"input distribution\" mean at line 184?\n\n3. What are the evidence that support the statement of \"distribution drift\" and \"distributional consisntency\"?  (See Weakness 3 for more details)\n\n4. Does the performance gain result from force feedback or force control during inference?\n\n5. How to generate the ground-truth of the external force output for the residual policy?\n\n6. Does the policy generalize better to novel scene configuration than policies using position-based control?", "limitations": "Yes", "rating": 4, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 2, "significance": 4, "originality": 3, "strengths_and_weaknesses": "Strengths:\n\n1. This paper proposes a novel and interesting approach, that collects correction data with compliance control, on DAgger.  The system allows more accurate, gentle and instantaneous corrections to the base policy.\n\n2. This paper clearly illustrates the categories and their corresponding limitations of prior arts.  Offline methods fail to cover failure scenarios, while online methods using position-based control introduct non-smooth behaviors in the correction data.  Reads can easily appreciate the idea of using compliance control for data collection.\n\n3. The manipulation tasks considered in this paper involve more complex physical interaction than typical pick-and-place tasks, which necessitates the usage of compliance control.\n\n---\n\nWeaknesses:\n\n1. This paper does not provide an explicit formulation of \"compliance control\".  Without solid background knowledge in control, I can only guess the action exectuion of robot policies is softer and smoother than position-based control.  However, I'm not sure if the system desires a small and constant external force during action execution or imposes an upper bound on the external force.\n\n2. The authors abuse the term--\"distribution\", throughout the paper.  For example, the authors refer to \"state distribution\" at line 22, while referring to \"action distribution\" at line 26.  With multiple meanings of the same word, it becomes challenging to read and understand the paper.  For example, what does \"the policy’s original distribution\" at line 107 mean? what does \"distribution shift\" mean at line 116? what does \"input distribution\" mean at line 184?\n\n3. The writing of this paper is often vague and imprecise.  For example:\n\n    - The paragraph of **distribution shift** is inconclusive.  Since both smooth and non-smooth correction behaviors deviate from the original action distribution of the base policy, CR-DAgger does not resolve the challenge of distribution drift.  There are other reasons that make CR-DAgger more effective than position-based DAgger.  Describing the problem without addressing it is weird.\n    - At line 40, the authors said \"*Unlike take-over corrections that may cause force discontinuity, our design allows smooth transition between correction/no correction mode, while maintaining distributional consistency with the original policy.*\".  However, I'm not sure what consistency is maintained by CR-DAgger.\n\n    These statements are not supported by any analysis.\n\n4. This paper does not ablate the effectiveness of force feedback.  Does the performance gain result from force feedback or force control during inference?\n\n5. This paper lacks technical details in the generation of the ground-truth of external force output for the residual policy.  The force sensor only keep tracks of the total force exerted on the sensor.\n\n6. This paper does not discuss the limitation that the human teleoperator is required to hold the teleoperation interface during the deployment of policies.  Such a teleoperation interface requires more labor effort than teleoperation via mouses or keyboards.\n\n7. This paper does not discuss the generalization of the compliance-control residual policy to novel scenarios.  Conditioning on force feedback, does the policy generalize better to novel scene configuration than policies using position-based control?\n\n8. I'm afraid this paper hinders the real power of compliance control.  Going beyond data correction, probably we should collect data with compliance control and train policies with force control, rather than those with position-based control, for imitation learning.", "was_revised": true, "final_justification": "The rebuttal has addressed my concerns in writing.", "rebuttal": "> Q1: Explicit formulation of \"compliance control\"... I'm not sure if the system desires a small and constant external force during action execution or imposes an upper bound on the external force.\n\nWe will add an introduction of compliance control to the paper. Compliance control is a traditional control technique to make a robot behave like a spring when experiencing external forces. It lets robots interact with the environment safely without creating huge forces. With compliance, the robot retreats when it experiences external forces, larger force leads to bigger position deviations. When there is no external force, compliance control tracks the target position exactly just like position-based control. So it does not require or “desire a certain type of external force”, and does not “impose bounds” on those forces. Rather, it responds continuously to the external force. Compliance control is a standard technique widely adopted in the manufacturing industry. The “admittance control” we used is one popular way to implement compliance control. A good reference is the “Force Control” chapter of the Handbook of Robotics [1].\n\n[1] Villani, Luigi, and Joris De Schutter. \"Force control.\" Springer handbook of robotics. Cham: Springer International Publishing, 2016. 195-220.\n\n> Q2: The authors abuse the term--\"distribution\"... \"state distribution\" at line 22, while referring to \"action distribution\" at line 26... what does \"the policy’s original distribution\" at line 107 mean? what does \"distribution shift\" mean at line 116? what does \"input distribution\" mean at line 184?\n\nThose “distribution” all refer to “the complete state-action distribution generated by a policy”, where the policy can be different depending on the context. In the final submission, we will standardize our terminology as follows:\n- “Policy distribution” refers to the joint distribution of states and actions encountered when deploying a specific policy.\n- “Distribution shift” refers to the mismatch between the state-action distributions in base policy training data and correction data.\n- “Input/state distribution” refers to the distribution of observations (states) the policy receives.\n\n> Q3: The writing is often vague and imprecise.\nThe paragraph of distribution shift is inconclusive...what consistency is maintained by CR-DAgger.\n\nThe paragraphs of concern describe the improvements of data quality from using our Compliant Intervention Interface, compared with the traditional “take over” correction. \nWe validated the improvements of data quality from our Compliant Intervention Interface by comparing policies trained with both our correction data [On-Policy Delta] and [Take-Over-Correction] data. Policies trained with our data generally perform better, as shown in Fig.6 and discussed in Section 4.4 Finding 3.\n\nWe agree that there could be more direct analysis of the data quality improvements, including less distribution shift and force discontinuity. We will add more analysis in the revised paper:\n\n- **[On-Policy Delta] introduces less distribution shift.**  We added a figure on our website that shows the distribution of fingertip trajectories across all dimensions in base policy training data, [On-Policy Delta] and [Take-Over-Correction] data. It demonstrates that [On-Policy Delta] data’s distribution is better aligned with base policy training data’s distribution than [Take-Over-Correction] data.\n- **[On-Policy Delta] enables smoother trajectories.** We also added figures comparing velocity magnitude within 1.5s of the corrections starts/ends to our project website. [On-Policy Delta] velocity magnitudes are smaller and more consistent, [Take-Over Correction] has notably larger magnitude and variations, demonstrating that [On-Policy Delta] encourages smoother trajectories.\n\nTo further clarify, compliant Intervention Interface allows humans to directly apply forces to the robot when the base policy is continuously running to correct its behavior. Firstly, humans are applying delta actions on top of base policy actions, instead of overwriting them. Additionally, the human demonstrator can always sense the policy’s intention through haptic feedback, and easily control the magnitude of delta actions by the amount of force applied to the handle. As a result, the interface limits humans from providing large corrections to the base policy, ensuring the state-action pairs encountered during correction are close to the state-action distribution generated by the base policy, thus maintaining distributional consistency with it.\n\n> Q4: Does not ablate the effectiveness of force feedback. Does the performance gain result from force feedback or force control during inference?\n\nThere are a few ways to interpret the question:\n\n“Does the performance improvement come from adding force feedback or from human correction?”\n\nWe already showed that adding force feedback significantly improves the residual policy (comparison between [Residual] and [Compliant Residual] in Fig. 6; Section 4.4 Finding 2).\nWe also added another baseline [Base Policy with Force] which added force input to the base policy, and showed that our [Compliant Residual] policy improves [Base Policy] more effectively than [Base Policy with Force], as shown in the results below (on the belt assembly task). This demonstrates the contribution of human correction in addition to force. \n\n| Method | Success Rate |\n|--------|--------------|\n| Base Policy | 15.625% |\n| Base Policy with Force | 43.75% |\n| Residual Policy | 28.125% |\n| Compliant Residual Policy | 96.825% |\n\n“The Compliant Residual Policy takes force feedback, and outputs force command. Does the performance gain result from only one of them?”\n\nBoth force feedback and force command are important components of our proposed method. We want to argue that both are useful and it does not provide much insight to ablate one of them. If our system does not have force feedback, then it is not possible to compute the force command label to train a policy with force command. If we do not use force command, then the position command alone cannot fully describe the demonstrator’s intention. \n\n“You use a force controller to run the experiments. Maybe the performance gain comes from this force controller, not your learned policy.”\n\nWe use force control (admittance control) for all policies being evaluated, as position control would immediately trigger the emergency stop in these contact-rich tasks and break the object or robot due to the large force exerted.\n\n> Q5: The generation of the ground-truth of external force output for the residual policy. The force sensor only keep tracks of the total force exerted on the sensor.\n\nThe total force exerted on the additional force sensor is exactly the external contact force as described in Section 3.1 “Correction recording with buttons and force sensor”, since the handle is mounted behind the sensor so human force would not affect its reading.\n\n> Q6: Limitation that the human teleoperator is required to hold the teleoperation interface during the deployment of policies... requires more labor effort than teleoperation via mouses or keyboards.\n\nWe agree that our kinesthetic teaching style correction may require more labor than teleoperation-based correction. We will add this to the limitation section. However, we want to point out that this is not a major concern because:\n- Additional labor is only needed when collecting correction data. Once the residual policy is trained, the robot can execute the task independently with a high success rate. The handle is also detachable and can be easily removed before evaluation.\n- Kinesthetic teaching provides key benefits including allowing intuitive and gentle correction, recording of human intended contact forces, and direct haptic feedback for the demonstrator. None of these benefits are available in traditional teleoperation systems with mouses or keyboards.\n\n> Q7: The generalization of the compliant residual policy to novel scenarios.\n\nThe book flipping task is evaluated with both known and novel books, as shown in Figure 4 (b). The test books differ significantly with train books in terms of size, weight, and appearance. We plan to also add another task of cable routing where we will test with several novel cables with OOD visual appearance, length, and stiffness.\n\n> Q8: This paper hinders the real power of compliance control. Going beyond data correction, we should collect data with compliance control and train policies with force control, rather than those with position-based control.\n\nThat is a good point. In fact, we already used compliance control when collecting even the base policy data. Compliance control is the reason why we can drag the robot around (kinesthetic teaching) to do demonstrations.\n\nHowever, current robot datasets and pretrained policies lack force modality, while our approach offers an easy solution to make such position-only policies force-aware with a small amount of correction data. Moving forward, while we hope that more and more public robot datasets and pretrained policies can include the force modality by default, we highlight that our approach is still able to provide performance improvements in the initial absence of force information by integrating it successfully through a residual prediction. \n\n> Q9: Formulation of the compliance control?\n\nThe compliance controller we use is an admittance controller, meaning that at every time step it reads the force feedback $f$ then compute a position command $p$, such that the force-position relationship behave like a spring-mass-damper system:\n$$f = m \\ddot p - \\sigma \\dot p + (p - p_0)k$$\nWhere $m$, $\\sigma$, $k$ are the inertia, damping, and stiffness parameters, respectively. $p_0$ is the resting position of the virtual spring. During policy rollout, $p_0$ is set to the base policy output position. We will add a section in the appendix to briefly introduce compliance control."}, {"review_id": "KaE7cmUKbJ", "reviewer": "Reviewer_NN22", "summary": "The paper introduces a residual policy to learn the human force correction during robot demonstration collection, which is verified to be effective in challenging contact-rich tasks over existing Dagger methods. This seems to be an efficient way to collect demonstrations, with an additional human handle and force sensor required.", "questions": "The base policy (pose) and residual policy (force) are learning two types of signals, how are these two combined into the admittance controller? Would there be both the impedance control and admittance control? Will these raise some requirements on robotic control systems that can be hard to work for all popular robots?\n\nFinetuning base policy is unstable, how about adding additional regularization (e.g. KL) with base policy as reference?", "limitations": "yes", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 4, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths \n\nThe paper is well written and easy to follow.\n\nThe motivation is well explained, that conventional off-policy correction can suffer from distribution shifts and additional errors. \n\nThe compliant intervention interface and compliant residual policy is novel. \n\nThe alation on design choices like batch size and sampling strategy is insightful.\n\nBase policy should be able to accomplish most tasks without requiring detailed force control, the lightweight residual policy can be applied for contact-rich tasks in an efficient manner.\n\nDifferent training recipes are discussed with details revealed, which is insightful.\n\n\n\nweakness\n\nThe experiments can be conducted on more diverse contact-rich tasks.", "was_revised": true, "final_justification": "Thanks for the rebuttal and additional results. I will keep my score.", "rebuttal": "> Q1: The experiments can be conducted on more diverse contact-rich tasks.\n\nWe plan to add another experiment on a cable routing task. It involves threading a flexible cable through multiple clips. The task requires precise force control to avoid cable kinking while navigating tight spaces.\n\n> Q2: The base policy (pose) and residual policy (force) are learning two types of signals, how are these two combined into the admittance controller? Would there be both the impedance control and admittance control? Will these raise some requirements on robotic control systems that can be hard to work for all popular robots?\n\nThe base policy, the residual policy and the compliance controller run in separate threads. They each take required input directly from the sensors. The base policy and residual policy outputs are combined into target position and forces, and are passed to the compliance controller as inputs.\n\nImpedance control and admittance control are two methods to implement compliance control. The users are free to choose the controller that works with their robot. Robots with good position control accuracy (e.g., all industrial robots like ABB, Fanuc, Kuka, UR, etc.) can use admittance control.  Robots with low gear ratio (e.g., Unitree, ARx) can use impedance control. In this work we choose admittance control because we use an UR robot.\n\nWe will add a paragraph to the paper to explain our controller structure in more detail, and introduce compliance control, which is a standard technique widely adopted in the manufacturing industry.\n\n> Q3: Finetuning base policy is unstable, how about adding additional regularization (e.g. KL) with base policy as reference?\n\nWe thank the reviewer for the suggestion. We added a new baseline that finetunes the base policy with KL regularization. Our implementation follows DPOK [1]. The results on the belt assembly task are listed below:\n\n| Method | Success Rate |\n|--------|--------------|\n| Base Policy | 15.625% |\n| Retrain | 34.375% |\n| Finetune | 0% |\n| Finetune w/ KL | 3.125% |\n| Residual Policy | 28.125% |\n| Compliant Residual Policy | 96.825% |\n\nAlthough [Finetune w/ KL] slightly improved the success rate over [Finetune] and exhibited smoother and stabler behaviors, it still hurts the [Base Policy] performance by 12%, while [Compliant Residual Policy] demonstrates a much more significant improvement over [Base Policy] by 81.2%. We have also added the rollout videos to the project website.\n\n[1] Fan, Y., Watkins, O., Du, Y., Liu, H., Ryu, M., Boutilier, C., ... & Lee, K. (2023). Dpok: Reinforcement learning for fine-tuning text-to-image diffusion models. Advances in Neural Information Processing Systems, 36, 79858-79885."}, {"review_id": "rfyQlA27N2", "reviewer": "Reviewer_UqHu", "summary": "The authors propose CR-DAgger, a novel approach for improving robot manipulation policies in real-world contact-rich tasks. The key innovations include (1) a Compliant Intervention Interface, which is a kinesthetic, force-sensitive correction setup allowing human demonstrators to apply residual actions during policy execution via admittance control; (2) a Compliant Residual Policy, which is a lightweight residual policy trained only on correction data that augments a pretrained policy. It incorporates additional force feedback to generate corrective pose deltas and desired forces. The method is evaluated on two real-world manipulation tasks: booking flipping and belt assembly. Extensive experiments show the method significantly outperforms standard approaches like retraining and finetuning in terms of success rate and data efficiency.", "questions": "- Could the authors provide a more formal description of the learning paradigm for the Compliant Residual Policy? For instance, what is the exact loss function used? How are forces and pose predictions weighted during training?\n- Have the authors tested CR-DAgger in more unstructured unseen cases? For example, more variations on the size/CoM location of the books or switching the locations of the small and large pulleys instead of minor changes on camera poses.", "limitations": "The authors acknowledge limitations in section 5.", "rating": 4, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 2, "originality": 3, "strengths_and_weaknesses": "Strengths:\n- [Quality]: The system is well-engineered and experimentally validated in realistic scenarios. Real-world experiments with contact-rich tasks make this work practically meaningful. On-policy correction and residual learning are thoughtfully designed to overcome distributional mismatch issues in DAgger-based learning.\n- [Clarity]: The writing is generally clear and well-structured. The figures effectively convey the main concepts and results. The supplementary video help the reader understand the significance of each design decision and the empirical performance.\n- [Significance]: CR-DAgger improves upon base policies with very little data (~50 episodes), which is valuable for robotics applications.\n- [Originality]: The idea of corrections during policy execution combined with compliance control is natural. Adding force feedback as an additional input modality into a residual policy is well motivated.\n\nWeaknesses:\n- [Lack of Theoretical Depth]: While the system demonstrates promising empirical results, the manuscript lacks theoretical analysis or general principles that could guide its application to other domains, especially considering that NeurIPS is a learning conference. There is no formal characterization of when and why the proposed framework is expected to work. The learning formulation is only described in paragraph form with no mathematical formulations, which makes it difficult to understand the precise mechanics of the training procedure or how the loss functions are designed and optimized. The absence of technical rigor limits the work's contribution from a foundational standpoint.\n- [Limited Generalization Evidence]: The experimental validation is restricted to two tightly controlled tasks with well-defined contact conditions. From Figure 4 and 5, the evaluation settings appear closely matched to the training configurations, and there is little evidence that the learned policies generalize to more unstructured variations. The lack of variety in test setups makes it hard to asses the robustness of the method, especially in settings with novel object geometries or out-of-distribution visual inputs.\n- [Incremental Novelty]: The proposed system is a combination of well-established ideas: DAgger, residual learning, and compliance control. While the integration is thoughtful and practically valuable, the method's novelty lies primarily in system engineering rather than algorithmic innovations.\n- [Structural Clarity]: The paper suffers from a somewhat scattered presentation. Specifically, key motivations are mixed into multiple sections rather than being clearly motivated up front. This makes the manuscript lack of narrative cohesion.", "was_revised": true, "final_justification": "In the initial review, my main concerns are: 1. lack of technical rigor; 2. lack of evidence of generalization. The authors propose some revisions to address the lack of technical rigor in the rebuttal and promise additional experiments to show evidence of generalization. Given these, I lean towards acceptance, but I am not confident that all proposed revisions can be fully integrated, considering the amount of work involved. I have also adjusted my confidence level to reflect this.", "rebuttal": "> Q1: [Lack of Theoretical Depth]: lacks theoretical analysis or general principles. No formal characterization of when and why the proposed framework is expected to work. No mathematical formulations... mechanics of the training procedure or how the loss functions are designed and optimized.\n\n**Mathematical formulation**\n\nThe base policy takes in images $I_t$ and proprioceptions $P_t$, and predicts $T_a$ frames of end-effector poses $A_t^p=(A_{t_0}^p, A_{t_1}^p, ..., A_{T_a}^p)$.\n\nThe Compliant Residual Policy takes in images $I_t$, proprioceptions $P_t$, and forces $F_t$, and predicts $T_r$ frames of delta poses $\\Delta A_t^p=(\\Delta A_{t_0}^p, \\Delta A_{t_0+1}^p, ..., \\Delta A_{t_0+{T_r}}^p)$ and target forces $A_t^f$.\n\nThe combined poses of $A_t^p$ and $\\Delta A_t^p$, and target forces $A_t^f$ are taken by an admittance controller to command the robot.\n\nThe admittance controller reads the force feedback $f$ then compute a position command $p$, such that the force-position relationship behave like a spring-mass-damper system:\n$$f = m \\ddot p - \\sigma \\dot p + (p - p_0)k$$\nWhere $m$, $\\sigma$, $k$ are the inertia, damping, and stiffness parameters, respectively. $p_0$ is the resting position of the virtual spring. During policy rollout, $p_0$ is set to the base policy output position.\n\n**Training procedure and loss functions**\n\nThe base policy is a UNet-based diffusion policy [1] that approximate the conditional distribution $p(A_t^p | I_t, P_t)$. It is formulated as a Denoising Diffusion Probabilistic Model (DDPM) [2], where the action generation is modeled as a denoising process (Stochastic Langevin Dynamics [3]). Starting from $A_t^{p, K}$ sampled from Gaussian noise, the DDPM performs K iterations of denoising $A_t^{p, K}, A_t^{p, K-1}, …, A_t^{p, 0}$, until a desired noise free final action $A_t^{p, 0}$. The process follows the equation:\n$$A_t^{p, k-1}=\\alpha(A_t^{p, k}-\\gamma\\epsilon_\\delta(I_t, P_t,A_t^{p, k}, k)+\\mathcal{N}(0,\\sigma^2 I) )$$ \nWhere $\\epsilon_\\delta$ is the noise prediction network that is optimized through learning, $\\mathcal{N}(0,\\sigma^2 I)$ is Gaussian noise added at each iteration, and $\\alpha, \\gamma, \\sigma$ are parameters determined by a noise scheduler [2].\n\nThe training loss of the base policy is:\n$$\\mathcal{L}=MSE(\\epsilon^k, \\epsilon_\\delta(I_t, P_t,A_t^0+\\epsilon^k, k))$$\nWhich is optimized through gradient descent.\n\nWe employ a MLP architecture for the Compliant Residual Policy, which is theoretically able to capture the full correction demonstration distribution, supported by the Universal Approximation Theorem [4] - a MLP with a single hidden layer can approximate any continuous function. We use a temporal encoding via casual convolution [5] as the force encoder, which helps capturing causal relations from sequential data like force [6].\n\nIts training loss is: \n$$\\mathcal{L}=MSE([\\Delta A_t^p, A_t^f], \\pi_{res}(I_t, P_t, F_t))$$\nWhich is optimized through gradient descent.\n\nWe will add these to the paper.\n\n**When and why the framework is expected to work**\n\nOur Compliant Residual Policy formulation allows additional modality being added to the policy.  Therefore, for contact-rich tasks where force is useful but may not be available in initial pretraining data, CR-DAgger is expected to work (Sec 4.4 Finding 2).\nAlso, when there are clear failure modes from the base policy, and human correction data can fully represent corrective behaviors with vision, position, and force, CR-DAgger can effectively learn such strategies (Sec 4.4 Finding 1).\n\n[1] Chi, C., Xu, Z., Feng, S., Cousineau, E., Du, Y., Burchfiel, B., ... & Song, S. (2023). Diffusion policy: Visuomotor policy learning via action diffusion. The International Journal of Robotics Research.\n\n[2] Ho J, Jain A and Abbeel P (2020) Denoising diffusion probabilistic models. arXiv preprint arXiv:2006.11239.\n\n[3] Welling M and Teh YW (2011) Bayesian learning via stochastic gradient langevin dynamics. In: Proceedings of the 28th international conference on machine learning.\n\n[4] Cybenko, G. (1989). Approximation by superpositions of a sigmoidal function. Mathematics of control, signals and systems.\n\n[5] Aaron van den Oord. “WaveNet: A Generative Model for Raw Audio”. In: arXiv preprint arXiv:1609.03499 (2016).\n\n[6] Hou, Y., Liu, Z., Chi, C., Cousineau, E., Kuppuswamy, N., Feng, S., ... & Song, S. (2024). Adaptive compliance policy: Learning approximate compliance for diffusion guided control. arXiv preprint arXiv:2410.09309.\n\n> Q2: [Limited Generalization Evidence]: The experimental validation is restricted to two tightly controlled tasks with well-defined contact conditions... there is little evidence that the learned policies generalize to more unstructured variations.\n\nThe book flipping task is evaluated with both known and novel books, as shown in Figure 4 (b). The test books differ significantly with train books in terms of size, weight, and appearance. We plan to also add another task of cable routing where we will test with several novel cables with OOD visual appearance, length, and stiffness.\n\nAlso, rather than having “well-defined contact conditions”, **we specifically choose tasks that  involve complex, multi-modal contact modes**. In the book flipping task, the initial finger insertion may touch the table in front of the book when there is a large enough gap, or it has to touch the front ridge of the book and push it back a little before touching down on the table. The insertion motion may or may not involve a contact between the fingertip and the back of the shelf wall, depending on the weight of the book and the policy strategy. The flipping motion after the rotation may simply be a pivoting of the book against the right shelf wall when the book is close to the right shelf wall; otherwise it must push the book to slide to the right, during which the book could slide off the fingers. In the belt task, the belt could get stuck on the base of the pulley or nearby gears/chains on the board when approaching the first pulley. The belt also may not move perfectly into the second pulley, instead, the following could happen: the finger may move down too early and hit the top of the pulley first; the finger may move too far away and stretch the belt too much so the belt ends up on the top of the pulley; the finger may move down too much and move the belt below the pulley slot. The loose side of the belt could get stuck on the wrong side of the pulley button.\n\nWe already showed the complexity of the tasks from the list of failure modes in the experiment section (Fig.4 and Fig.5). We will include a more complete list of failure cases and explanations in the final paper. \n\n> Q3: [Incremental Novelty]: The proposed system is a combination of well-established ideas: DAgger, residual learning, and compliance control... the method's novelty lies primarily in system engineering rather than algorithmic innovations.\n\nDAgger, residual learning, and compliance control are very general research directions that are still actively being explored, rather than “well-established ideas”. There are many valuable works that continue to improve these formulations or leverage these techniques in a new way to solve important robotics problems. We believe that building on top of existing ideas and expanding existing research directions do not devalue a paper.\n\nOur technical contributions represent fundamental novelties, not just engineering combinations:\n- We propose a compliant residual interface that allows collecting on-policy delta correction data with haptic feedback, which ensures smoothness of correction data while maintaining consistency between correction data distribution and policy induced state-action distribution.\n- We propose a novel compliant residual policy formulation that effectively learns correction strategies for contact-rich tasks and enables any position-only base policy to become force-aware through minimal correction data. We also demonstrate a novel insight that on-policy delta corrections and force modality enable stable residual learning.\n- We are the first to demonstrate that DAgger can work effectively for contact-rich tasks with as few as 50 correction demonstrations, while prior works on DAgger often overlook and struggle with contact-rich tasks.\n- We present a systematic study of intervention interfaces and policy update methods.\n\n> Q4: [Structural Clarity]: The paper suffers from a somewhat scattered presentation. Specifically, key motivations are mixed into multiple sections rather than being clearly motivated up front.\n\nWe present high-level key motivations in the introduction: “How to collect informative human correction data” and “How to effectively update the policy with new data”. Then in the method section, we motivate each of our system/algorithm design choices before introducing it. We appreciate any detailed suggestions from the reviewer regarding how specific sections and areas of the paper could be adjusted to improve the overall clarity of the manuscript.\n\n> Q5: Could the authors provide a more formal description of the learning paradigm for the Compliant Residual Policy? For instance, what is the exact loss function used? How are forces and pose predictions weighted during training?\n\nThe formulation is answered in Q1.\nDuring training, the joint pose and force predictions are normalized across each channel dimension and weighted equally.\n\n> Q6: Test in more unstructured unseen cases? More variations on the size/CoM location of the books or switching the locations of the small and large pulleys instead of minor changes on camera poses.\n\nAs explained in the answer to Q2, our experiment setup does involve variations of item/setup positions as well as novel items not seen during training. \n\nRegarding “minor changes on camera poses”, we would like to clarify that all our experiments use a consistent single wrist-mounted camera for policy observation, which moves dynamically with the robot end-effector."}]

Author rebuttal:

Meta-review: This paper addresses Dataset Aggregation (DAgger) for real-world contact-rich manipulation, an important problem where traditional approaches struggle with force dynamics and physical interaction complexities. The method introduces a novel Compliant Intervention Interface using compliance control for smooth human corrections and a Compliant Residual Policy that integrates force feedback.

Reviewers praised the clear motivation, well-engineered system design, and impressive experimental results. Some found the compliant intervention interface novel and the significant performance gains on challenging real-world tasks compelling.

Reviewers raised concerns about lack of mathematical rigor, unclear terminology, insufficient generalization analysis, and the need to isolate contributions of force modality versus human corrections. Some questioned whether the work represented algorithmic innovation versus primarily system engineering.

The authors largely addressed these concerns in their rebuttal, providing mathematical formulations, adding key ablation studies, clarifying terminology, and proposing an experiment for broader generalization evaluation.

One concern that may not have been fully addressed is Reviewer J472's question regarding generalization comparison to baselines—the authors' rebuttal under Q7 primarily highlights what generalization their experiments require rather than providing comparative evidence.

Finally, the authors violated the rebuttal policy by updating their project website with additional images and videos, but I'd recommend leniency given that this did not appear to materially affect the reviews.
Decision: Accept (poster)