  Learning competitive behaviors in multi-agent settings such as racing
requires long-term reasoning about potential adversarial interactions. This
paper presents Deep Latent Competition (DLC), a novel reinforcement learning
algorithm that learns competitive visual control policies through self-play in
imagination. The DLC agent imagines multi-agent interaction sequences in the
compact latent space of a learned world model that combines a joint transition
function with opponent viewpoint prediction. Imagined self-play reduces costly
sample generation in the real world, while the latent representation enables
planning to scale gracefully with observation dimensionality. We demonstrate
the effectiveness of our algorithm in learning competitive behaviors on a novel
multi-agent racing benchmark that requires planning from image observations.
Code and videos available at
https://sites.google.com/view/deep-latent-competition.
