BASE TTS: Lessons from building a billion-parameter Text-to-Speech model
  on 100K hours of data
Authors: name: Mateusz Łajszczak; name: Guillermo Cámbara; name: Yang Li; name: Fatih Beyhan; name: Arent van Korlaar; name: Fan Yang; name: Arnaud Joly; name: Álvaro Martín-Cortinas; name: Ammar Abbas; name: Adam Michalski; name: Alexis Moinet; name: Sri Karlapati; name: Ewa Muszyńska; name: Haohan Guo; name: Bartosz Putrycz; name: Soledad López Gambino; name: Kayeon Yoo; name: Elena Sokolova; name: Thomas Drugman
Published: 2024-02-12 22:21:30

Abstract: We introduce a text-to-speech (TTS) model called BASE TTS, which stands for
Big Adaptive Streamable TTS with
Emergent abilities. BASE TTS is the largest TTS model to-date,
trained on 100K hours of public domain speech data, achieving a new
state-of-the-art in speech naturalness. It deploys a 1-billion-parameter
autoregressive Transformer that converts raw texts into discrete codes
("speechcodes") followed by a convolution-based decoder which converts these
speechcodes into waveforms in an incremental, streamable manner. Further, our
speechcodes are built using a novel speech tokenization technique that features
speaker ID disentanglement and compression with byte-pair encoding. Echoing the
widely-reported "emergent abilities" of large language models when trained on
increasing volume of data, we show that BASE TTS variants built with 10K+ hours
and 500M+ parameters begin to demonstrate natural prosody on textually complex
sentences. We design and share a specialized dataset to measure these emergent
abilities for text-to-speech. We showcase state-of-the-art naturalness of BASE
TTS by evaluating against baselines that include publicly available large-scale
text-to-speech systems: YourTTS, Bark and TortoiseTTS. Audio samples generated
by the model can be heard at https://amazon-ltts-paper.com/.