Octo: An Open-Source Generalist Robot Policy
Authors: name: Octo Model Team; name: Dibya Ghosh; name: Homer Walke; name: Karl Pertsch; name: Kevin Black; name: Oier Mees; name: Sudeep Dasari; name: Joey Hejna; name: Tobias Kreiman; name: Charles Xu; name: Jianlan Luo; name: You Liang Tan; name: Pannag Sanketi; name: Quan Vuong; name: Ted Xiao; name: Dorsa Sadigh; name: Chelsea Finn; name: Sergey Levine
Published: 2024-05-20 17:57:01

Abstract: Large policies pretrained on diverse robot datasets have the potential to
transform robotic learning: instead of training new policies from scratch, such
generalist robot policies may be finetuned with only a little in-domain data,
yet generalize broadly. However, to be widely applicable across a range of
robotic learning scenarios, environments, and tasks, such policies need to
handle diverse sensors and action spaces, accommodate a variety of commonly
used robotic platforms, and finetune readily and efficiently to new domains. In
this work, we aim to lay the groundwork for developing open-source, widely
applicable, generalist policies for robotic manipulation. As a first step, we
introduce Octo, a large transformer-based policy trained on 800k trajectories
from the Open X-Embodiment dataset, the largest robot manipulation dataset to
date. It can be instructed via language commands or goal images and can be
effectively finetuned to robot setups with new sensory inputs and action spaces
within a few hours on standard consumer GPUs. In experiments across 9 robotic
platforms, we demonstrate that Octo serves as a versatile policy initialization
that can be effectively finetuned to new observation and action spaces. We also
perform detailed ablations of design decisions for the Octo model, from
architecture to training data, to guide future research on building generalist
robot models.