TRANSIC: Sim-to-Real Policy Transfer by Learning from Online Correction
Authors: name: Yunfan Jiang; name: Chen Wang; name: Ruohan Zhang; name: Jiajun Wu; name: Li Fei-Fei
Published: 2024-05-16 17:59:07

Abstract: Learning in simulation and transferring the learned policy to the real world
has the potential to enable generalist robots. The key challenge of this
approach is to address simulation-to-reality (sim-to-real) gaps. Previous
methods often require domain-specific knowledge a priori. We argue that a
straightforward way to obtain such knowledge is by asking humans to observe and
assist robot policy execution in the real world. The robots can then learn from
humans to close various sim-to-real gaps. We propose TRANSIC, a data-driven
approach to enable successful sim-to-real transfer based on a human-in-the-loop
framework. TRANSIC allows humans to augment simulation policies to overcome
various unmodeled sim-to-real gaps holistically through intervention and online
correction. Residual policies can be learned from human corrections and
integrated with simulation policies for autonomous execution. We show that our
approach can achieve successful sim-to-real transfer in complex and
contact-rich manipulation tasks such as furniture assembly. Through synergistic
integration of policies learned in simulation and from humans, TRANSIC is
effective as a holistic approach to addressing various, often coexisting
sim-to-real gaps. It displays attractive properties such as scaling with human
effort. Videos and code are available at https://transic-robot.github.io/