  We introduce environment predictive coding, a self-supervised approach to
learn environment-level representations for embodied agents. In contrast to
prior work on self-supervised learning for images, we aim to jointly encode a
series of images gathered by an agent as it moves about in 3D environments. We
learn these representations via a zone prediction task, where we intelligently
mask out portions of an agent's trajectory and predict them from the unmasked
portions, conditioned on the agent's camera poses. By learning such
representations on a collection of videos, we demonstrate successful transfer
to multiple downstream navigation-oriented tasks. Our experiments on the
photorealistic 3D environments of Gibson and Matterport3D show that our method
outperforms the state-of-the-art on challenging tasks with only a limited
budget of experience.
