A Generalist Dynamics Model for Control
Authors: name: Ingmar Schubert, name: Jingwei Zhang, name: Jake Bruce, name: Sarah Bechtle, name: Emilio Parisotto, name: Martin Riedmiller, name: Jost Tobias Springenberg, name: Arunkumar Byravan, name: Leonard Hasenclever, name: Nicolas Heess

We investigate the use of transformer sequence models as dynamics models
(TDMs) for control. In a number of experiments in the DeepMind control suite,
we find that first, TDMs perform well in a single-environment learning setting
when compared to baseline models. Second, TDMs exhibit strong generalization
capabilities to unseen environments, both in a few-shot setting, where a
generalist model is fine-tuned with small amounts of data from the target
environment, and in a zero-shot setting, where a generalist model is applied to
an unseen environment without any further training. We further demonstrate that
generalizing system dynamics can work much better than generalizing optimal
behavior directly as a policy. This makes TDMs a promising ingredient for a
foundation model of control.