^RFLAV: Rolling Flow matching for infinite Audio Video generation
Authors: name: Alex Ergasti; name: Giuseppe Gabriele Tarollo; name: Filippo Botti; name: Tomaso Fontanini; name: Claudio Ferrari; name: Massimo Bertozzi; name: Andrea Prati
Published: 2025-03-11 11:18:47

Abstract: Joint audio-video (AV) generation is still a significant challenge in
generative AI, primarily due to three critical requirements: quality of the
generated samples, seamless multimodal synchronization and temporal coherence,
with audio tracks that match the visual data and vice versa, and limitless
video duration. In this paper, we present , a novel transformer-based
architecture that addresses all the key challenges of AV generation. We explore
three distinct cross modality interaction modules, with our lightweight
temporal fusion module emerging as the most effective and computationally
efficient approach for aligning audio and visual modalities. Our experimental
results demonstrate that  outperforms existing state-of-the-art models
in multimodal AV generation tasks. Our code and checkpoints are available at
https://github.com/ErgastiAlex/R-FLAV.