VibeVoice Technical Report
Authors: name: Zhiliang Peng, name: Jianwei Yu, name: Wenhui Wang, name: Yaoyao Chang, name: Yutao Sun, name: Li Dong, name: Yi Zhu, name: Weijiang Xu, name: Hangbo Bao, name: Zehua Wang, name: Shaohan Huang, name: Yan Xia, name: Furu Wei
Organization: Microsoft Research

This report presents VibeVoice, a novel model designed to synthesize
long-form speech with multiple speakers by employing next-token diffusion,
which is a unified method for modeling continuous data by autoregressively
generating latent vectors via diffusion. To enable this, we introduce a novel
continuous speech tokenizer that, when compared to the popular Encodec model,
improves data compression by 80 times while maintaining comparable performance.
The tokenizer effectively preserves audio fidelity while significantly boosting
computational efficiency for processing long sequences. Thus, VibeVoice can
synthesize long-form speech for up to 90 minutes (in a 64K context window
length) with a maximum of 4 speakers, capturing the authentic conversational
``vibe'' and surpassing open-source and proprietary dialogue models.