Google scientists published a research paper introducing Lumiere on January 23rd, 2024. The paper, titled "Lumiere: A Space-Time Diffusion Model for Video Generation" is filed under Computer Vision and Pattern Recognition - a nod to the multimodality of the model. It's not just text models that can be multimodal! A multimodal AI video model could accept inputs of text, images, audio samples, or even other videos (perhaps a combination of two or more) to inform the generated video.
The researchers who authored the paper include Omer Bar-Tal, Hila Chefer, Omer Tov, Charles Herrmann, Roni Paiss, Shiran Zada, Ariel Ephrat, Junhwa Hur, Yuanzhen Li, Tomer Michaeli, Oliver Wang, Deqing Sun, Tali Dekel, and Inbar Mosseri.