U ni C o M : A Universal Code-Switching Speech Generator (2025)

Abstract: Code-switching (CS), the alternation between two or more languages within a single speaker’s utterances, is common in real-world conversations and poses significant challenges for multilingual speech technology. However, systems capable of handling this phenomenon remain underexplored, primarily due to the scarcity of suitable datasets. To resolve this issue, we propose Universal Code-Mixer (UniCoM), a novel pipeline for generating high-quality, natural CS samples without altering sentence semantics. Our approach utilizes an algorithm we call Substituting WORDs with Synonyms (SWORDS), which generates CS speech by replacing selected words with their translations while considering their parts of speech. Using UniCoM, we construct Code-Switching FLEURS (CS-FLEURS), a multilingual CS corpus designed for automatic speech recognition (ASR) and speech-to-text translation (S2TT). Experimental results show that CS-FLEURS achieves high intelligibility and naturalness, performing comparably to existing datasets on both objective and subjective metrics. We expect our approach to advance CS speech technology and enable more inclusive multilingual systems.

## introduction multilingual speech technology has advanced rapidly in recent years. early research primarily focused on similar language families @xcite or a small range of languages @xcite , but the latest developments over the recent few years have significantly increased model sophistication. researchers have scaled models @xcite to support over 100 languages directly, or more than 1,000 languages @xcite through language-specific modules @xcite . however, most approaches are designed for monolingual utterances and struggle to process speech containing multiple mixed languages effectively. in particular, code-switching (cs), the practice of alternating between languages within a conversation, is prevalent in multilingual communities. while cs-asr has been widely studied, most efforts have centered on major languages (e.g., arabic @xcite , indic @xcite , mandarin @xcite , paired with english), leaving research on broader languages relatively underexplored. in comparison, research on cs-s2tt remains at an early stage, with recent efforts @xcite . given this landscape, we identify two key questions: (1) why has cs research lagged behind multilingual speech technologies? and (2) why are existing studies confined to a few major languages? we attribute these limitations primarily to the scarcity and accessibility of cs datasets. to elaborate, cs occurs at both the inter-sentential (between sentences) and intra-sentential (within a sentence) levels @xcite , and constructing datasets for these cases, particularly intra-sentential cs with diverse language pairs, requires a substantial number of fluent multilingual speakers. however, their rarity and the high cost of data collection pose significant challenges. in this context, there are two potential data augmentation methods to address these challenges: cs speech generation using a multilingual textto-speech (tts) system @xcite or combining existing monolingual sources @xcite . the first approach generates natural cs samples by preserving speaker identity across utterances. however, scaling this method is challenging due to the complexities of massively multilingual tts, such as resource imbalance and prosodic variation. the second approach synthesizes cs samples from abundant monolingual data but may lose linguistic nuances and speaker identity when utterances from different languages are simply concatenated, degrading the quality of intra-sentential cs. to address these limitations, we propose universal code-mixer (unicom), capable of generating intra-sentential cs samples across diverse languages while maintaining both contextual relevance and speaker consistency. unicom consists of three stages: preprocessing, source mixing, and style unification. in the preprocessing stage, potential artifacts are removed from the input sources to ensure the stability of the pipeline. the source mixing stage introduces substituting words with synonyms (swords), a novel algorithm to generate speech-text pairs for intra-sentential cs. finally, the style unification stage standardizes speaker styles across different sources. consequently, building on unicom, we introduce cs-fleurs: a comprehensive multilingual cs corpus encompassing 253 language pairs-marking a significant advancement over traditional single language pair datasets. furthermore, the inclusion of over 70 n-way parallel sentences offers potential for future exploration in cs-s2tt tasks. experimental results demonstrate that cs-fleurs achieves comparable intelligibility and naturalness to human-generated cs datasets. we anticipate our approach will serve as a foundation for generalizable cs speech technology, paving the way for broader applications. our contributions are summarized as follows: ## unicom in this section, we propose universal code-mixer (unicom), a novel code-switching speech data generation pipeline. unicom is a universal and generalized framework designed for broad applicability across a wide range of languages. unlike previous methods that generate inter-sentential codeswitching samples by simply concatenating sentences from two different languages, unicom enables intra-sentential code-switching speech generation while preserving the unique linguistic features of each language, such as phrase structure and word order. moreover, unicom preserves speaker identity by unifying the speaker's style, making the generated speech more reflective of naturally occurring code-switching in real-life scenarios. in the following sections, we provide a detailed description of each step in the proposed framework. ## cs-fleurs this section describes cs-fleurs, an intrasentential cs speech dataset spanning 253 language pairs, constructed using unicom. overall metadata of cs-fleurs is illustrated in ## experiments in this section, we evaluate the validity of cs-fleurs to prove the effectiveness of unicom. all experiments were performed on two rtx 3090 gpus with 24gb vram each, and subjective metrics were evaluated through a user study involving a total of 42 participants and over 2,500 utterances. ## conclusion in this paper, we introduce unicom, a universal pipeline for code-switching speech generation. specifically, we propose a swords algorithm to facilitate the generation of intra-sentential codeswitching samples while maintaining the natural meaning of the original speech. our pipeline then employs voice conversion to ensure the naturalness of generated samples by unifying speaker styles across utterances without compromising intelligibility. finally, we release cs-fleurs, a massively multilingual code-switching speech dataset designed for the cs-asr task while offering future utility for cs-s2tt scenarios. in experiments, cs-fleurs exhibits high intelligibility and nat-uralness compared to human-generated datasets, while contributing to the improvements in cs-asr training and performance. we believe our work will serve as a cornerstone for a more generalized code-switching speech technology in the future.