  As learning-based approaches progress towards automating robot controllers
design, transferring learned policies to new domains with different dynamics
(e.g. sim-to-real transfer) still demands manual effort. This paper introduces
SimGAN, a framework to tackle domain adaptation by identifying a hybrid physics
simulator to match the simulated trajectories to the ones from the target
domain, using a learned discriminative loss to address the limitations
associated with manual loss design. Our hybrid simulator combines neural
networks and traditional physics simulaton to balance expressiveness and
generalizability, and alleviates the need for a carefully selected parameter
set in System ID. Once the hybrid simulator is identified via adversarial
reinforcement learning, it can be used to refine policies for the target
domain, without the need to collect more data. We show that our approach
outperforms multiple strong baselines on six robotic locomotion tasks for
domain adaptation.
