Modularcomputing/dynamic-accent-preview

Dynamic Accent: Code-switching TTS

Code-switching TTS for Nigerian languages. English, Nigerian Pidgin, Igbo, Yoruba and Hausa, mixed freely inside one sentence and spoken by one consistent voice, with no model swapping between languages.

Dynamic Accent is a fine-tune of FunAudioLLM/CosyVoice2-0.5B by Modular. It adds:

• per-token language embeddings, a speaker token and an adversarial (gradient-reversal) speaker classifier on the LM, so the voice stays the same across a language switch (after Zhang et al., Google, 2019, Learning to Speak Fluently in a Foreign Language);
• training on about 178 h of Nigerian-language and code-switched speech, including about 102 h of Igbo;
• learned [pause]/[breath] events and a speaking-rate instruction (Phase 3);
• a flow-matching decoder fine-tuned on the anchor voice for tone and pitch detail.

It is a research release, not a production system. Read Limitations before you use it.

Versions and voices

Two model versions ship side by side. Pick one with --version, or render both with --compare and A/B before use.

version	LM	flow	decoding	what changed
phase3 (default)	checkpoints/phase3/p3_lm_full (Phase 2 LM + 5,000 steps on a 178 h re-cut with ~102 h Igbo, [pause]/[breath] tags and a speed instruction, rate normal)	shared checkpoints/phase2/p2_flow_sft	one pass per paragraph, silence-aware repetition check, no audio post-processing	lower CER in Igbo/Hausa/Yoruba/code-switch/paragraphs, comma pauses at the anchor's length, clipped endings 63% → 11%
phase2	checkpoints/phase2/p2_lm_B (~120 h, pause profile B)	checkpoints/phase2/p2_flow_sft	sentence by sentence, internal silences capped at 0.6 s	the Phase 2 final system

voice (--voice)	who	notes
anchor_male (default)	WAXAL Yoruba speaker 4, male	the voice both versions were trained around
waxal_ibo_5_female	WAXAL Igbo speaker 5, female, native Igbo	zero-shot (prompt clip, no fine-tune); best CER of the three on the female test subset
waxal_yor_5_female	WAXAL Yoruba speaker 5, female, native Yoruba	zero-shot; strongest identity match, but its 3.8 s prompt has no pauses, so pauses are short and endings often clipped

Recommended pairing (author's listening verdict): Phase 3 sounds best with the female voices; Phase 2 is fine for the male anchor. infer.py prints a note when you pick another pairing. The female voices were evaluated with phase3 only; they run with phase2 but are untested there. Settings live in settings.json.

Setup (single GPU)

# 1. CosyVoice code (the commit the model was built and tested on)
git clone --recursive https://github.com/FunAudioLLM/CosyVoice.git
cd CosyVoice && git checkout 074ca6dc9e80a2f424f1f74b48bdd7d3fea531cc && git submodule update --init --recursive && cd ..

# 2. This repo
huggingface-cli download Modularcomputing/dynamic-accent-preview --local-dir dynamic-accent-preview

# 3. Python deps (Python 3.10; on Colab keep the preinstalled torch/torchaudio)
pip install -r dynamic-accent-preview/requirements.txt

The stock base model FunAudioLLM/CosyVoice2-0.5B (tokenizer, vocoder, speech tokenizer, speaker encoder) downloads to pretrained/CosyVoice2-0.5B on first run; nothing in it needs changing. Point --model_dir (or COSYVOICE2_DIR) at an existing copy, and --cosyvoice_dir (or COSYVOICE_REPO) at your CosyVoice clone. A 16 GB GPU is enough for the CLI; the Gradio app keeps both versions loaded. Tested on an H100 (bf16). infer.py also runs on CPU in fp32, which is slow. T4-class GPUs fall back to fp16, which has not been tested.

Usage

cd dynamic-accent-preview
# default: --version phase3 --voice anchor_male
python infer.py --cosyvoice_dir ../CosyVoice --text "<eng>this is dynamic accent,</eng> <pcm>e dey sound nice abi?</pcm>" --out out.wav

# recommended pairing
python infer.py --cosyvoice_dir ../CosyVoice --version phase3 --voice waxal_ibo_5_female \
  --text "<ibo>Kedu ka ị mere?</ibo> <pcm>Abeg no vex, I go call you for evening.</pcm>" --out ibo_f.wav

# A/B: the same text, seed and voice through both versions -> cmp_phase2.wav, cmp_phase3.wav
python infer.py --cosyvoice_dir ../CosyVoice --compare --voice anchor_male --text "..." --out cmp.wav

python infer.py --list_voices     # voices, versions, recommended pairing

flag	what it does
--text	the text: tagged (below) or plain with --auto_tag
--version phase2|phase3	model version (default phase3)
--voice anchor_male|waxal_ibo_5_female|waxal_yor_5_female	voice (default anchor_male); old ids anchor, waxal_ibo_5, waxal_yor_5 still work
--list_voices	list voices, versions and the recommended pairing
--compare	render with both versions → <out>_phase2.wav, <out>_phase3.wav
--rate default|none|very slow|slow|normal|fast|very fast	Phase 3 speaking-rate instruction to the LM (default: the version's setting, normal). It is a learned instruction, not time-stretching. phase2 ignores it
--auto_tag	tag untagged mixed text automatically (see below), print the tagged text, then synthesise
--tag_only	with --auto_tag: print the tagged text and stop, so you can edit it and pass it back as --text
--force_lang eng|pcm|ibo|yor|hau	read the whole text in one language (tags are ignored)
--lang_map "omo=pcm,sha=pcm"	per-word language overrides for the tagger
--lid none|glotlid	with --auto_tag: add GlotLID v3 clause scores (Apache-2.0 plus notices, 1.7 GB, pip install fasttext)
--seed N	sampling seed (default 0). Try 2–3 seeds and keep the take you like
--out file.wav	output path (24 kHz WAV)
--dtype auto|bf16|fp16|fp32	LM autocast dtype on GPU (samples: bf16). CPU always runs fp32
--stitch	phase3: sentence by sentence instead of one pass (not recommended)
--prompt_clip 0-3	which anchor prompt clip (released systems: 0)
--model_dir, --cosyvoice_dir, --lm, --flow, --prompts, --voices	paths and checkpoint overrides
--pause_mode clause, --speed	phase2 only: the experimental "prosody v2" (fixed gaps plus WSOLA speed-up). A listening test rejected it; the defaults (model, 1.0) are the real Phase 2 system

Generation on GPU is not bit-reproducible: bf16 sampling can pick a different token on near-ties, so the same seed can give a slightly different take.

Controlling code-switching

1. Explicit tags (most control). Wrap each span in <eng>, <pcm>, <ibo>, <yor> or <hau>. Keep punctuation inside the span it belongs to and put a space between spans. Untagged text is read as English.

<eng>this is dynamic accent,</eng> <pcm>e dey sound nice abi?</pcm>
<pcm>omo,</pcm> <eng>I'm very excited to announce this new model,</eng> <ibo>ọ na-eme m obi ụtọ,</ibo> <pcm>I no go lie give you</pcm>
<yor>Ẹ káàárọ̀ o, ṣé dáadáa ni?</yor> <eng>I will send you the report this afternoon.</eng>
<hau>Ina kwana, yaya aiki?</hau> <eng>The meeting has been moved to Friday morning.</eng>
<ibo>Ndewo nụ, ụmụnna m.</ibo> <pcm>Una well done o, I know say the road no easy.</pcm> <eng>We have three things to discuss this evening.</eng>
<eng>Please tell everyone that</eng> <yor>a ó pàdé ní ọ̀la.</yor>

A switch can fall mid-sentence (<eng>Please tell everyone that</eng> <yor>a ó pàdé ní ọ̀la.</yor>). The tag sets the pronunciation and accent of the words inside it; the voice stays the same.

2. --auto_tag for untagged text. tagger.py does word- and span-level language ID: Yoruba/Igbo/Hausa letters (ẹ ṣ / ị ụ ṅ / ɓ ɗ ƙ ƴ), small lexicons of frequent words, Pidgin markers ("dey", "abi", "omo", "wetin", "abeg", "una", "no go", "e don", "know say", pronoun + "go" + verb …) and clause context for ambiguous words ("na", "o", "ka", "ya"). It prints the tagged text so you can check and edit it:

python infer.py --auto_tag --tag_only --text "omo, I'm very excited to announce this new model, ọ na-eme m obi ụtọ, I no go lie give you"
# [tagged] <pcm>omo,</pcm> <eng>I'm very excited to announce this new model,</eng> <ibo>ọ na-eme m obi ụtọ,</ibo> <pcm>I no go lie give you</pcm>
python infer.py --auto_tag --text "..." --out out.wav            # tag, print, synthesise
python tagger.py "Kedu ka ị mere? Abeg no vex, I go call you for evening."   # tagger on its own

Write Yoruba and Igbo with their diacritics where you can; text without diacritics is much harder to identify. See the tagger accuracy below.

3. --force_lang reads everything in one language, e.g. --force_lang ibo.

4. --lang_map fixes words the tagger gets wrong, e.g. --lang_map "omo=pcm,sha=pcm,oga=pcm".

Gradio app (local)

python app.py --cosyvoice_dir ../CosyVoice      # http://127.0.0.1:7860  (--share for a public link)

It has a text box, an auto-tag toggle with an editable tagged-text preview, word overrides, a force-language option, version, voice and rate dropdowns, a seed, and a Compare button that plays Phase 2 and Phase 3 side by side.

From Python

import types, soundfile as sf, infer
a = types.SimpleNamespace(model_dir="pretrained/CosyVoice2-0.5B", cosyvoice_dir="../CosyVoice", lm="", flow="",
                          prompts="prompts/prompts.json", voices="prompts/voices.json", dtype="auto", stitch=False,
                          rate="default", voice="waxal_ibo_5_female")
tts = infer.build("phase3", a)
wav = tts.say("<ibo>Kedu ka ị mere?</ibo> <eng>The meeting has been moved to Friday morning.</eng>", seed=0)
sf.write("out.wav", wav, tts.sr)

Samples

Seed 0, rendered with infer.py. Inputs: para_ibo_pcm_eng, para_yor_eng, para_hau_eng_pcm (code-switched paragraphs), ibo_000 (Igbo), custom_dynacc ("this is dynamic accent, e dey sound nice abi?") and omo_announce ("omo, I'm very excited to announce this new model, ọ na-eme m obi ụtọ, I no go lie give you").

input	Phase 2, anchor_male	Phase 3, anchor_male	Phase 3, female
para_ibo_pcm_eng	mp3	mp3	waxal_ibo_5_female
para_yor_eng	mp3	mp3	waxal_yor_5_female
para_hau_eng_pcm	mp3	mp3	–
ibo_000	mp3	mp3	waxal_ibo_5_female
custom_dynacc	mp3	mp3	waxal_yor_5_female
omo_announce	mp3	mp3	waxal_ibo_5_female, waxal_yor_5_female

Older renders: samples/pcm_eng_dynamic_accent_seed{0,1,2}.mp3 (Phase 2, three seeds) and samples/prosody_v2/ (the rejected prosody-v2 post-processing, kept for reference).

Evaluation: Phase 2 vs Phase 3

183-item eval set (30 per language, 10 code-switched, 3 hand-written + 20 native long-form paragraphs), anchor voice, seed 0. CER is ASR round-trip: MMS-1b-all for Igbo/Hausa/Yoruba, Whisper-large-v3-turbo for English and Pidgin, the lower of the two for code-switched items and paragraphs. "No tones" (in brackets) strips diacritics. SIM is ECAPA cosine to the anchor (ceiling about 0.84 on real held-out anchor clips). The Pidgin judge is weak: real Pidgin recordings score about 0.43.

system	ibo	hau	yor	pcm	eng	CS	para (hand / native)	SIM
phase2 as shipped (sentence stitching)	0.215 (0.171)	0.194	0.234 (0.149)	0.196	0.035	0.163	0.192 / 0.154	0.785
Phase 2 LM, Phase 3 decoding	0.243 (0.189)	0.169	0.275 (0.175)	0.195	0.046	0.185	0.195 / 0.158	0.787
phase3	0.219 (0.173)	0.151	0.233 (0.145)	0.202	0.043	0.151	0.170 / 0.155	0.788

Same decoding, Phase 3 LM vs Phase 2 LM: lower CER in every language except Pidgin, same speaker similarity.

system	comma pause s	sentence pause s	pauses / s	pause-length W1 vs anchor	trailing silence s	clipped endings (< 0.1 s)	articulation syl/s
real anchor recordings	0.417	0.543	0.624	0	0.16	15%	4.43
Phase 2 LM, Phase 3 decoding	0.403	0.389	0.329	0.194	0.08	63%	4.36
phase3	0.422	0.453	0.319	0.161	0.17	11%	4.44

Female voices (29-item subset, phase3):

voice	CER	no-tone CER	SIM to the speaker's own real clips	clipped endings
anchor_male (same items)	0.182	0.141	0.12–0.13 (it is a different speaker)	17%
waxal_ibo_5_female	0.149	0.101	0.64	21%
waxal_yor_5_female	0.171	0.134	0.76	62%

Naturalness has not been rated by native speakers (no MOS or blind A/B yet). The numbers above are automatic proxies.

Auto-tagger accuracy

Word-level accuracy of tagger.py against hand-tagged text (no GlotLID / with --lid glotlid):

test set	words	rule-based	+ GlotLID	exact sentences (rule-based)
the 6 sample inputs above	145	1.000	1.000	6/6
10 new mixed sentences, written before the lexicons	113	0.991	0.991	9/10
12 code-switched eval items (overlaps the samples)	206	0.995	1.000	11/12
150 single-language eval sentences (real WAXAL/FLEURS text, never looked at while building the lexicons)	1,697	0.898	0.915	115/150

On the held-out single-language set, per language (rule-based / + GlotLID): English 0.99 / 1.00, Igbo 1.00 / 1.00, Yoruba 1.00 / 1.00, Hausa 0.89 / 0.94, Pidgin 0.63 / 0.65. Pidgin written without markers looks like English and gets tagged eng. The mixed test sentences were written and gold-tagged by the tagger's author, not native speakers, so their scores are optimistic. Check the printed tags, and use explicit tags or --lang_map when it matters.

Limitations

• Pace: speech is still 13–28% slower than native speakers (Igbo 4.60 vs 5.30 syl/s, Hausa 3.77 vs 5.26 syl/s), and there are about half as many pauses per second as in real speech. It can sound careful rather than fully native.
• Pidgin is the weakest language: only about 5 h of training data, no reliable ASR judge, and the auto-tagger's Pidgin/English split is its weakest point (~0.63 word accuracy on plain Pidgin).
• No laughter or other expressive events: the training data has effectively no laughter, so tags like [laughter] do nothing useful. [pause]/[breath] were learned from data, but there is no reliable user control over them yet.
• English after a Nigerian-language span keeps a strong local accent.
• Yoruba tone quality needs native listening; the automatic tone metric is saturated.
• Female voices are zero-shot; waxal_yor_5_female copies its short prompt (short pauses, clipped endings).
• Not evaluated for numbers, abbreviations or URLs; write numbers as words.
• Do not use it to impersonate real people. The voices are WAXAL Media Trust speakers, released under CC-BY-SA-4.0 for speech technology.

What is in the repo

path	what it is	licence
checkpoints/phase3/p3_lm_full/	Phase 3 LM (best.pt, val step 4,500), config.json (rate, sampler, one-pass, points at the shared Phase 2 flow), spk2id.json, args.json, log.jsonl	CC-BY-SA-4.0
checkpoints/phase2/p2_lm_B/	Phase 2 LM (best.pt) and spk2id.json	CC-BY-SA-4.0
checkpoints/phase2/p2_flow_sft/best.pt	anchor-SFT flow-matching decoder, used by both versions	CC-BY-SA-4.0
prompts/	anchor prompt clips (clip0..3, prompts.json with transcripts and S3 tokens) and female voice prompts (voice_*.wav, voices.json), all from WAXAL	CC-BY-SA-4.0
samples/	example renders	CC-BY-SA-4.0
settings.json	versions, voices, recommended pairing	Apache-2.0
infer.py, tagger.py, app.py	inference CLI, auto-tagger, Gradio app	Apache-2.0
code/	the original training and synthesis code, for reference	Apache-2.0
requirements.txt, cosyvoice_requirements.txt	dependencies (cosyvoice_requirements.txt is CosyVoice's own file at the commit used)	–
assets/	banner	–

Training data and attribution

The model weights are trained on, and the voice prompts are taken from, the datasets below. Several are CC-BY-SA-4.0, so the weights, voice prompts and samples are released under CC-BY-SA-4.0 (share-alike).

dataset	used for	licence	attribution
WAXAL, Media Trust TTS subsets (Igbo, Yoruba, Hausa, Nigerian Pidgin)	main training data; the anchor voice (Yoruba speaker 4) and the female prompt voices (Igbo speaker 5, Yoruba speaker 5)	CC-BY-SA-4.0 (as listed on the dataset card for the Media Trust subsets)	Google and Media Trust, WAXAL (arXiv:2602.02734)
Open.Bible Igbo (Biblica® Open Igbo Contemporary Bible™, Audio Edition)	Igbo (Phase 3)	CC-BY-SA-4.0	Biblica, Inc. via open.bible; Biblica® and Open Igbo Contemporary Bible™ are trademarks of Biblica, Inc.
NaijaS2ST (Igbo)	Igbo, many native speakers (Phase 3)	CC-BY-4.0	McGill-NLP, NaijaS2ST (arXiv:2604.16287)
IgboAPI (ibo-dict, ibo-dict-expansion)	Igbo sentences (Phase 3)	CC-BY-4.0	Nkọwa okwu / IgboAPI contributors
IgboSynCorp	natural Igbo speech (Phase 3)	CC0-1.0	University of Ibadan and Afe Babalola University (Lacuna Fund)
FLEURS (English, Hausa, Igbo, Yoruba)	multi-speaker data	CC-BY-4.0	Google, FLEURS (Conneau et al., 2022)
BibleTTS (Yoruba, Hausa)	Yoruba and Hausa	CC-BY-SA-4.0	Meyer et al., BibleTTS (2022); Biblica / open.bible audio
OpenSLR SLR70 (Nigerian English)	English	CC-BY-SA-4.0	Google, crowdsourced Nigerian English
OpenSLR SLR86 (Yoruba)	Yoruba	CC-BY-SA-4.0	Google, crowdsourced Yoruba
code-switched pairs	about 9 h of synthetic code-switch training pairs	follows the source clips (CC-BY-SA-4.0 / CC-BY-4.0)	built by concatenating clips from the sets above, following the Google 2019 code-switching recipe (Zhang et al., 2019); no separate Google dataset
base model FunAudioLLM/CosyVoice2-0.5B	LM initialisation, flow, vocoder, tokenizers	Apache-2.0	FunAudioLLM / Alibaba Tongyi Lab, CosyVoice 2 (Du et al., 2024)
optional GlotLID (not bundled)	--lid glotlid in the tagger	Apache-2.0 plus notices	Kargaran et al., GlotLID (EMNLP 2023)

Excluded on licence grounds: WaZoBiaSpeech (bans TTS use), NaijaVoices (non-commercial), and non-commercial TTS weights.

Licence

• Model weights, voice prompts and samples (checkpoints/, prompts/, samples/): CC-BY-SA-4.0. You may use them commercially. You must credit Modular and the datasets listed above, and anything you derive from the weights (fine-tunes, merges) must be shared under CC-BY-SA-4.0 too.
• Code (infer.py, tagger.py, app.py, code/, settings.json): Apache-2.0.
• The base model, CosyVoice2-0.5B, is Apache-2.0; its notices still apply to the parts it contributes.

Citation

@misc{dynamicaccent2026,
  title  = {Dynamic Accent: Code-switching TTS for Nigerian languages},
  author = {Modular},
  year   = {2026},
  url    = {https://huggingface.co/Modularcomputing/dynamic-accent-preview}
}