Video Occupancy Models
Authors: name: Manan Tomar; name: Philippe Hansen-Estruch; name: Philip Bachman; name: Alex Lamb; name: John Langford; name: Matthew E. Taylor; name: Sergey Levine
Published: 2024-06-25 17:57:38

Abstract: We introduce a new family of video prediction models designed to support
downstream control tasks. We call these models Video Occupancy models (VOCs).
VOCs operate in a compact latent space, thus avoiding the need to make
predictions about individual pixels. Unlike prior latent-space world models,
VOCs directly predict the discounted distribution of future states in a single
step, thus avoiding the need for multistep roll-outs. We show that both
properties are beneficial when building predictive models of video for use in
downstream control. Code is available at
https://github.com/manantomar/video-occupancy-models{github.com/manantomar/video-occupancy-models}.