  Challenging manipulation tasks can be solved effectively by combining
individual robot skills, which must be parameterized for the concrete physical
environment and task at hand. This is time-consuming and difficult for human
programmers, particularly for force-controlled skills. To this end, we present
Shadow Program Inversion (SPI), a novel approach to infer optimal skill
parameters directly from data. SPI leverages unsupervised learning to train an
auxiliary differentiable program representation ("shadow program") and realizes
parameter inference via gradient-based model inversion. Our method enables the
use of efficient first-order optimizers to infer optimal parameters for
originally non-differentiable skills, including many skill variants currently
used in production. SPI zero-shot generalizes across task objectives, meaning
that shadow programs do not need to be retrained to infer parameters for
different task variants. We evaluate our methods on three different robots and
skill frameworks in industrial and household scenarios. Code and examples are
available at https://innolab.artiminds.com/icra2021.
