Emu Video: Factorizing Text-to-Video Generation by Explicit Image
  Conditioning
Authors: name: Rohit Girdhar, name: Mannat Singh, name: Andrew Brown, name: Quentin Duval, name: Samaneh Azadi, name: Sai Saketh Rambhatla, name: Akbar Shah, name: Xi Yin, name: Devi Parikh, name: Ishan Misra

We present Emu Video, a text-to-video generation model that factorizes the
generation into two steps: first generating an image conditioned on the text,
and then generating a video conditioned on the text and the generated image. We
identify critical design decisions--adjusted noise schedules for diffusion, and
multi-stage training--that enable us to directly generate high quality and high
resolution videos, without requiring a deep cascade of models as in prior work.
In human evaluations, our generated videos are strongly preferred in quality
compared to all prior work--81% vs. Google's Imagen Video, 90% vs. Nvidia's
PYOCO, and 96% vs. Meta's Make-A-Video. Our model outperforms commercial
solutions such as RunwayML's Gen2 and Pika Labs. Finally, our factorizing
approach naturally lends itself to animating images based on a user's text
prompt, where our generations are preferred 96% over prior work.