SkillMimic-V2: Learning Robust and Generalizable Interaction Skills from
  Sparse and Noisy Demonstrations
Authors: name: Runyi Yu, name: Yinhuai Wang, name: Qihan Zhao, name: Hok Wai Tsui, name: Jingbo Wang, name: Ping Tan, name: Qifeng Chen

We address a fundamental challenge in Reinforcement Learning from Interaction
Demonstration (RLID): demonstration noise and coverage limitations. While
existing data collection approaches provide valuable interaction
demonstrations, they often yield sparse, disconnected, and noisy trajectories
that fail to capture the full spectrum of possible skill variations and
transitions. Our key insight is that despite noisy and sparse demonstrations,
there exist infinite physically feasible trajectories that naturally bridge
between demonstrated skills or emerge from their neighboring states, forming a
continuous space of possible skill variations and transitions. Building upon
this insight, we present two data augmentation techniques: a Stitched
Trajectory Graph (STG) that discovers potential transitions between
demonstration skills, and a State Transition Field (STF) that establishes
unique connections for arbitrary states within the demonstration neighborhood.
To enable effective RLID with augmented data, we develop an Adaptive Trajectory
Sampling (ATS) strategy for dynamic curriculum generation and a historical
encoding mechanism for memory-dependent skill learning. Our approach enables
robust skill acquisition that significantly generalizes beyond the reference
demonstrations. Extensive experiments across diverse interaction tasks
demonstrate substantial improvements over state-of-the-art methods in terms of
convergence stability, generalization capability, and recovery robustness.