Title: Compliant Residual DAgger: Improving Real-World Contact-Rich Manipulation with Human Corrections
Venue: NeurIPS 2025
Accepted: yes
Keywords: robot manipulation

imitation learning

human-in-the-loop learning
Abstract: We address key challenges in Dataset Aggregation (DAgger) for real-world contact-
rich manipulation: how to collect informative human correction data and how to
effectively update policies with this new data. We introduce Compliant Residual
DAgger (CR-DAgger), which contains two novel components: 1) a Compliant
Intervention Interface that leverages compliance control, allowing humans to pro-
vide gentle, accurate delta action corrections without interrupting the ongoing
robot policy execution; and 2) a Compliant Residual Policy formulation that learns
from human corrections while incorporating force feedback and force control.
Our system significantly enhances performance on precise contact-rich manipu-
lation tasks using minimal correction data, improving base policy success rates
by over 60% on two challenging tasks (book flipping and belt assembly) while
outperforming both retraining-from-scratch and finetuning approaches. Through
extensive real-world experiments, we provide practical guidance for implementing
effective DAgger in real-world robot learning tasks.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe authors propose CR-DAgger, a novel approach for improving robot manipulation policies in real-world contact-rich tasks. The key innovations include (1) a Compliant Intervention Interface, which is a kinesthetic, force-sensitive correction setup allowing human demonstrators to apply residual actions during policy execution via admittance control; (2) a Compliant Residual Policy, which is a lightweight residual policy trained only on correction data that augments a pretrained policy. It incorporates additional force feedback to generate corrective pose deltas and desired forces. The method is evaluated on two real-world manipulation tasks: booking flipping and belt assembly. Extensive experiments show the method significantly outperforms standard approaches like retraining and finetuning in terms of success rate and data efficiency.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n- [Quality]: The system is well-engineered and experimentally validated in realistic scenarios. Real-world experiments with contact-rich tasks make this work practically meaningful. On-policy correction and residual learning are thoughtfully designed to overcome distributional mismatch issues in DAgger-based learning.\n- [Clarity]: The writing is generally clear and well-structured. The figures effectively convey the main concepts and results. The supplementary video help the reader understand the significance of each design decision and the empirical performance.\n- [Significance]: CR-DAgger improves upon base policies with very little data (~50 episodes), which is valuable for robotics applications.\n- [Originality]: The idea of corrections during policy execution combined with compliance control is natural. Adding force feedback as an additional input modality into a residual policy is well motivated.\n\nWeaknesses:\n- [Lack of Theoretical Depth]: While the system demonstrates promising empirical results, the manuscript lacks theoretical analysis or general principles that could guide its application to other domains, especially considering that NeurIPS is a learning conference. There is no formal characterization of when and why the proposed framework is expected to work. The learning formulation is only described in paragraph form with no mathematical formulations, which makes it difficult to understand the precise mechanics of the training procedure or how the loss functions are designed and optimized. The absence of technical rigor limits the work's contribution from a foundational standpoint.\n- [Limited Generalization Evidence]: The experimental validation is restricted to two tightly controlled tasks with well-defined contact conditions. From Figure 4 and 5, the evaluation settings appear closely matched to the training configurations, and there is little evidence that the learned policies generalize to more unstructured variations. The lack of variety in test setups makes it hard to asses the robustness of the method, especially in settings with novel object geometries or out-of-distribution visual inputs.\n- [Incremental Novelty]: The proposed system is a combination of well-established ideas: DAgger, residual learning, and compliance control. While the integration is thoughtful and practically valuable, the method's novelty lies primarily in system engineering rather than algorithmic innovations.\n- [Structural Clarity]: The paper suffers from a somewhat scattered presentation. Specifically, key motivations are mixed into multiple sections rather than being clearly motivated up front. This makes the manuscript lack of narrative cohesion.\n\nQUESTIONS:\n- Could the authors provide a more formal description of the learning paradigm for the Compliant Residual Policy? For instance, what is the exact loss function used? How are forces and pose predictions weighted during training?\n- Have the authors tested CR-DAgger in more unstructured unseen cases? For example, more variations on the size/CoM location of the books or switching the locations of the small and large pulleys instead of minor changes on camera poses.\n\nLIMITATIONS:\nThe authors acknowledge limitations in section 5.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 2, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-06-16"}

comments: SUMMARY:
The paper introduces a residual policy to learn the human force correction during robot demonstration collection, which is verified to be effective in challenging contact-rich tasks over existing Dagger methods. This seems to be an efficient way to collect demonstrations, with an additional human handle and force sensor required.

STRENGTHS AND WEAKNESSES:
Strengths 

The paper is well written and easy to follow.

The motivation is well explained, that conventional off-policy correction can suffer from distribution shifts and additional errors. 

The compliant intervention interface and compliant residual policy is novel. 

The alation on design choices like batch size and sampling strategy is insightful.

Base policy should be able to accomplish most tasks without requiring detailed force control, the lightweight residual policy can be applied for contact-rich tasks in an efficient manner.

Different training recipes are discussed with details revealed, which is insightful.



weakness

The experiments can be conducted on more diverse contact-rich tasks.

QUESTIONS:
The base policy (pose) and residual policy (force) are learning two types of signals, how are these two combined into the admittance controller? Would there be both the impedance control and admittance control? Will these raise some requirements on robotic control systems that can be hard to work for all popular robots?

Finetuning base policy is unstable, how about adding additional regularization (e.g. KL) with base policy as reference?

LIMITATIONS:
yes
TITLE: Official Review
CLARITY: 4
IMPACT: 3
ORIGINALITY: 3
DATE: 2025-07-03

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper tackles the challenge in collecting correction data for robot policies.  Prior works are mostly based on two approaches: (1) observe-the-collect, an offline method that re-collect successful trajectories after observing failed operation of robots, and (2) take-over-correction, an online method that introduce human's interference when failures are about to occur.  However, these approaches are limited, as the former fails to cover the the policy’s typical failure scenarios and the later introduces non-smooth transitions or errorneous correction due to spatial mismatch and teleoperationlatency.  This paper proposes an online alternative that leverages compliance control to provide smooth and instantaneous action corrections.  Moreover, this paper introduces a compliant residual policy that predicts residual actions and intended force control for the base policy.  Carrying out experiments on real-world manipulation tasks, this paper demonstrations significant performance gain over position-control baselines.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n\n1. This paper proposes a novel and interesting approach, that collects correction data with compliance control, on DAgger.  The system allows more accurate, gentle and instantaneous corrections to the base policy.\n\n2. This paper clearly illustrates the categories and their corresponding limitations of prior arts.  Offline methods fail to cover failure scenarios, while online methods using position-based control introduct non-smooth behaviors in the correction data.  Reads can easily appreciate the idea of using compliance control for data collection.\n\n3. The manipulation tasks considered in this paper involve more complex physical interaction than typical pick-and-place tasks, which necessitates the usage of compliance control.\n\n---\n\nWeaknesses:\n\n1. This paper does not provide an explicit formulation of \"compliance control\".  Without solid background knowledge in control, I can only guess the action exectuion of robot policies is softer and smoother than position-based control.  However, I'm not sure if the system desires a small and constant external force during action execution or imposes an upper bound on the external force.\n\n2. The authors abuse the term--\"distribution\", throughout the paper.  For example, the authors refer to \"state distribution\" at line 22, while referring to \"action distribution\" at line 26.  With multiple meanings of the same word, it becomes challenging to read and understand the paper.  For example, what does \"the policy’s original distribution\" at line 107 mean? what does \"distribution shift\" mean at line 116? what does \"input distribution\" mean at line 184?\n\n3. The writing of this paper is often vague and imprecise.  For example:\n\n    - The paragraph of **distribution shift** is inconclusive.  Since both smooth and non-smooth correction behaviors deviate from the original action distribution of the base policy, CR-DAgger does not resolve the challenge of distribution drift.  There are other reasons that make CR-DAgger more effective than position-based DAgger.  Describing the problem without addressing it is weird.\n    - At line 40, the authors said \"*Unlike take-over corrections that may cause force discontinuity, our design allows smooth transition between correction/no correction mode, while maintaining distributional consistency with the original policy.*\".  However, I'm not sure what consistency is maintained by CR-DAgger.\n\n    These statements are not supported by any analysis.\n\n4. This paper does not ablate the effectiveness of force feedback.  Does the performance gain result from force feedback or force control during inference?\n\n5. This paper lacks technical details in the generation of the ground-truth of external force output for the residual policy.  The force sensor only keep tracks of the total force exerted on the sensor.\n\n6. This paper does not discuss the limitation that the human teleoperator is required to hold the teleoperation interface during the deployment of policies.  Such a teleoperation interface requires more labor effort than teleoperation via mouses or keyboards.\n\n7. This paper does not discuss the generalization of the compliance-control residual policy to novel scenarios.  Conditioning on force feedback, does the policy generalize better to novel scene configuration than policies using position-based control?\n\n8. I'm afraid this paper hinders the real power of compliance control.  Going beyond data correction, probably we should collect data with compliance control and train policies with force control, rather than those with position-based control, for imitation learning.\n\nQUESTIONS:\n1. What is the formulation of the compliance control?\n\n2. What does \"the policy’s original distribution\" at line 107 mean? What does \"distribution shift\" mean at line 116? What does \"input distribution\" mean at line 184?\n\n3. What are the evidence that support the statement of \"distribution drift\" and \"distributional consisntency\"?  (See Weakness 3 for more details)\n\n4. Does the performance gain result from force feedback or force control during inference?\n\n5. How to generate the ground-truth of the external force output for the residual policy?\n\n6. Does the policy generalize better to novel scene configuration than policies using position-based control?\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 5, "CLARITY": 2, "IMPACT": 4, "ORIGINALITY": 3, "DATE": "2025-07-03"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper presents a data-efficient method for improving pretrained policies on contact-rich robot manipulation tasks - by collecting delta action correction data and force feedback from human, and using this data to train a force-aware residual policy. The data collection method employs admittance control for human operators to injecting necessary intervention online while the robot follows base policy actions (10Hz). The collected data is used to train a residual policy that predicts delta action and autoregressively predict expected force (50Hz), which operates on top of the base policy in deployment. \n\nThe paper conducts extensive experiments and studies on two real-world tasks. By comparing between various ablations and baselines, the authors show that the proposed strategy significantly outperform other existing strategies.\n\nSTRENGTHS AND WEAKNESSES:\n## Strength\n\n- I enjoy reading this paper for its clarity and transparency. The problem it studies is well motivated, and the proposed methods address the problem well, backed by sufficient experimental results.\n- Incorporating force feedback for human correction and policy deployment in contact-rich tasks is a foundamental problem in robot manipulation. I'm glad to see the authors propose a viable solution that effectively improves the success rate.\n- Many of the discussions are informative and insightful. In particular, I like how the method is compared with different correction data collection strategies, as well as different training methods.\n\n## Weakness\n\n- There are two aspects that may explain the performance gain introduced by the complaint residual policy on top of the base policy: one is the human correction, and another is the force modality. By reading the paper, it's not clear to me how these two aspects contribute to the success of complaint residual policy. It would be very helpful if the authors provide more comparison on this, for example, training the base policy with force input as well and compare its results with complaint residual policy.\n\nQUESTIONS:\n- How is the frequency of the compliant residual policy to be selected? WIth executed without the residual policy, does the admittance control run at the same frequency as well?\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 4, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-07"}