B ahasa Harmony: A Comprehensive Dataset for B ahasa Text-to-Speech Synthesis with Discrete Codec Modeling of E n G en- TTS . (2024)

Abstract: This research introduces a comprehensive Bahasa text-to-speech (TTS) dataset and a novel TTS model, EnGen-TTS, designed to enhance the quality and versatility of synthetic speech in the Bahasa language. The dataset, spanning 55.00 hours and 52K audio recordings, integrates diverse textual sources, ensuring linguistic richness. A meticulous recording setup captures the nuances of Bahasa phonetics, employing professional equipment to ensure high-fidelity audio samples. Statistical analysis reveals the dataset’s scale and diversity, laying the foundation for model training and evaluation. The proposed EnGen-TTS model performs better than established baselines, achieving a Mean Opinion Score (MOS) of 4.45 ± 0.13. Additionally, our investigation on real-time factor and model size highlights EnGen-TTS as a compelling choice, with efficient performance. This research marks a significant advancement in Bahasa TTS technology, with implications for diverse language applications.

## introduction the bahasa language, spoken by a vibrant and diverse community, serves as a linguistic tapestry that encapsulates the rich cultural heritage of its speakers. in our increasingly digital world, the demand for advanced speech synthesis technologies tailored specifically to bahasa becomes more pronounced. this necessity arises from the need for synthetic speech that authentically captures the nuances of bahasa expressions, accommodating the linguistic diversity within the bahasa-speaking population, including various dialects, registers, and cultural nuances. existing text-to-speech (tts) systems, while making strides in the broader landscape, often fall short in addressing the requirements of bahasa. this gap underscores the motivation for our research, which introduces a meticulously curated bahasa tts dataset and an innovative tts model, engen-tts. while other models have attempted to address the synthesis challenges for diverse languages, including bahasa, they may exhibit drawbacks such as limited adaptability, linguistic richness, or efficiency. our proposed engen-tts model not only fills these gaps but also showcases superior performance when compared to established baselines. the model achieves a remarkable mean opinion score (mos) of 4.45 ± 0.13, outperforming existing models even without fine-tuning on additional bahasa data. our key strength is, positioning engen-tts as a solution for high-quality, adaptive text-to-speech synthesis across various languages. ## dataset in creating our bahasa text-to-speech (tts) dataset, we curated a linguistically diverse textual foundation. this dataset is integral to our innovative tts model, engen-tts. drawing from sources like wikipedia and incorporating content from chat-gpt translation, our approach involved a strategic gathering of text samples. this fusion of varied linguistic contexts lays the groundwork for a robust bahasa tts dataset, capturing the language's nuanced breadth of expression. ## architecture our research introduces the engen-tts, a novel bahasa text-to-speech synthesis system inspired by the state-of-the-art encodec-based tts bark @xcite .the system leverages the architectural framework illustrated in figure 1: 1. m-t5 encoder: a frozen multi-li t5 encoder is utilized for conditioning on text latents. this encoder is pre-trained and is kept frozen during training. audio codec setting: we adopt the pre-trained encodec model @xcite as our tokenizer, a convolutional encoder-decoder model handling 22050 hz audio at variable bitrates. the encoder generates embeddings at 75 hz for 22050 khz input, reducing the sampling rate by 320 times. these embeddings use residual vector quantization (rvq) with 4 hierarchical quantizers of 1024 entries each, corresponding to a 3k bitrate for audio reconstruction at 22050 hz.bfor our purposes, we use only the first entity of the 750 × 4 discrete representation matrix, as it contains all phonetic and content information, resulting in a 750 × 1 matrix. higher bitrates, such as 12k, require more quantizers (e.g., 16) and offer better reconstruction quality. the encodec decoder then reconstructs the waveform at 22050 hz from the discrete codes. ## conclusions in conclusion, our study presents a pivotal advancement in bahasa text-to-speech (tts) synthesis, combining a richly curated dataset with a groundbreaking model design. our comprehensive bahasa tts dataset, encompassing over 55 hours of audio and 52k recording, is a robust resource, crafted with inputs from proficient voice artists and varied textual content. the introduced model, engen-tts, excels in performance, surpassing conventional benchmarks with its innovative architecture, which includes a multi-task t5 (m-t5) encoder and a neural codec language modeling module, without necessitating extra fine-tuning for bahasa. this design not only enhances speech synthesis quality but also ensures computation efficiency, establishing a new standard in tts technology. our work not only pushes forward the boundaries of bahasa tts but also lays the groundwork for future developments in multilingual text-to-speech systems, promising high-quality and diverse linguistic applications.