A Generalist Dynamics Model for Control
Authors: name: Ingmar Schubert; name: Jingwei Zhang; name: Jake Bruce; name: Sarah Bechtle; name: Emilio Parisotto; name: Martin Riedmiller; name: Jost Tobias Springenberg; name: Arunkumar Byravan; name: Leonard Hasenclever; name: Nicolas Heess
Published: 2023-05-18 12:14:49

Abstract: We investigate the use of transformer sequence models as dynamics models
(TDMs) for control. In a number of experiments in the DeepMind control suite,
we find that first, TDMs perform well in a single-environment learning setting
when compared to baseline models. Second, TDMs exhibit strong generalization
capabilities to unseen environments, both in a few-shot setting, where a
generalist model is fine-tuned with small amounts of data from the target
environment, and in a zero-shot setting, where a generalist model is applied to
an unseen environment without any further training. We further demonstrate that
generalizing system dynamics can work much better than generalizing optimal
behavior directly as a policy. This makes TDMs a promising ingredient for a
foundation model of control.