gabar-tech/cosyvoice2-amharic

CosyVoice2 Amharic

This release fine-tunes the text-to-token and token-to-mel stages of CosyVoice2-0.5B (FunAudioLLM / Alibaba, Apache-2.0) for Amharic. It keeps the base model's reference-audio voice cloning, which uses about ten seconds of audio. The model produces intelligible Amharic; frequent hesitation sounds are its main quality limitation.

The tokenizer represents Ge'ez text without unknown tokens, but the stock checkpoint did not produce intelligible Amharic in our tests. The vocoder, speech tokenizer and speaker encoder are unchanged.

amharic_text.py is the text normalizer used for training labels and at inference.

Hear it

Each pair uses the same sentence, reference audio, settings and seed.

#	Test case	Stock CosyVoice2	+ Gabar fine-tune
1	Ordinary prose
2	Prose, ፥ punctuation
3	Prose (ejectives ቡ/ጅ)
4	Numbers + ዓ.ም. date abbreviation
5	ዶ/ር title abbreviation
6	Question intonation
7	Technical prose
8	Mixed punctuation + question

The players render on Hugging Face only. On GitHub the clips are in demo/ — stock: 1 · 2 · 3 · 4 · 5 · 6 · 7 · 8; fine-tuned: 1 · 2 · 3 · 4 · 5 · 6 · 7 · 8. The weights (llm.pt, flow.pt, ~2 GB) are on Hugging Face; everything else is mirrored on GitHub.

Texts: demo/sentences.txt. Reference voice: demo/reference.wav, a consenting member of the Gabar team (). Both models got the text after amharic_text.normalize, cross-lingual mode (reference audio only), seed 1234. No outputs were selected from multiple attempts. Known weaknesses are under Limitations.

Files in this release

Two files: llm.pt (the text → speech-token language model) and flow.pt (speech tokens → mel), both full weights, both fine-tuned from the stock CosyVoice2-0.5B checkpoint. You combine them with the stock model, which the loader downloads from FunAudioLLM/CosyVoice2-0.5B at the pinned revision; the HiFT vocoder, the speech tokenizer, the campplus speaker encoder and the Qwen text tokenizer are stock and unmodified.

This is not a standalone model and not an adapter: the two stages are complete replacement weights. This checkpoint is intended only for Amharic; use the original base checkpoint for its supported languages.

Amharic text front-end

amharic_text.py builds the training labels and the loader runs it on every input, so training and inference see the same text. It converts Ge'ez numerals, digits, percentages and times to words, expands about 100 abbreviations, collapses homophones, and reduces punctuation to ። ፣ ፤ ? !.

from amharic_text import normalize
normalize("ዶ/ር አበበ በ2018 ዓ.ም በተደረገው ምርጫ 75% ድምፅ አገኙ።")
# 'ዶክተር አበበ በሁለት ሺህ አስራ ስምንት ኣመተ ምህረት በተደረገው ምርጫ ሰባ አምስት በመቶ ድምጽ አገኙ።'

Training data

The training set contains the three sources listed below. Every clip's filename starts with its corpus prefix. We audited the assembled training directory before training; the report is included in this repository as is (audit/corpus_audit.txt). Both stages were trained from the stock CosyVoice2 checkpoint on exactly that directory.

prefix	source	licence	clips	hours
ih_	Our own studio recordings	ours	569	1.25
wxl_	WaxalNLP Amharic (Digital Umuganda / Google)	CC-BY-SA-4.0	40921	190.93
cv_	Common Voice Amharic	CC0-1.0	1055	1.45
	total		42545	193.63

WaxalNLP comes as 48 kHz and Common Voice as 32/48 kHz MP3; both were resampled to 24 kHz.

Licence: why CC-BY-SA-4.0

WaxalNLP is licensed under CC BY-SA 4.0. The fine-tuned weights are released under CC BY-SA 4.0. Third-party components retain their original licences; the base model is Apache-2.0. Credit to Digital Umuganda, the WaxalNLP contributors, and the Common Voice contributors.

Architecture

Base: CosyVoice2-0.5B, FunAudioLLM/CosyVoice2-0.5B at revision eec1ae6c79877dbd9379285cf8789c9e0879293d; code FunAudioLLM/CosyVoice at commit 074ca6dc9e80a2f424f1f74b48bdd7d3fea531cc. The loader pins these revisions because the fine-tuned stages depend on the corresponding tokenizer, vocoder, and speaker encoder.

component	treatment
LLM (Qwen-based text → speech-token, 0.5 B)	full fine-tune from the stock llm.pt; shipped
Flow (speech tokens → mel, flow matching)	full fine-tune from the stock flow.pt, non-streaming batches; shipped
HiFT vocoder (hift.pt)	stock, untouched, not shipped
Speech tokenizer (speech_tokenizer_v2.onnx)	stock, frozen (its quantization sets a ceiling on ejectives/gemination — see Limitations)
campplus speaker encoder / Qwen text tokenizer	stock, untouched, not shipped

Evaluation

Held-out set: we evaluated 100 clips sampled from a 104-clip speaker-disjoint held-out partition of the same corpus, split before training by a seeded rule, so whole speakers are held out and none of their sentences appear in training. Checked independently of the trainer's own split: HELD OUT: eval ∩ train = ∅ at clip, speaker and sentence level; all eval stems are cv_/ih_/wxl_. (audit/holdout_verify.txt). Both models ran on the same clips with the same per-clip reference audio, in cross-lingual mode (reference audio only — handing either model the reference transcript would let it copy the answer), through the same code: the released wrapper with our weights, and the identical assembly with the stock llm.pt/flow.pt.

metric	stock CosyVoice2	+ Gabar fine-tune
Amharic CER ↓ (Meta omniASR-CTC-3B)	0.939 (over 67/100 clips)	0.094 (over 99/100 clips)
UTMOS ↑ (naturalness MOS predictor)	2.177	2.787
ECAPA cosine ↑ (speaker similarity to reference)	0.397	0.758
generation failures (empty / <0.5 s / error)	0.0%	0.0%

Per-clip numbers: audit/eval/. CER method: Meta's stock omniASR-CTC-3B transcribes the generated audio; reference and hypothesis are homophone-collapsed and punctuation-stripped before comparison. Same ASR and normalization for both models. UTMOS was trained on English MOS ratings and is a rough signal only. Means cover clips that produced audio; generation failures are reported separately.

CER was calculated only where the ASR returned a usable transcript: 67 of 100 stock outputs and 99 of 100 fine-tuned outputs. Missing transcripts are excluded from the mean, so the coverage counts should be read alongside the CER values.

In our listening review: the stock outputs were not intelligible Amharic. The fine-tuned outputs were intelligible, but frequent "emm" and "uh" fillers made them sound unnatural.

Usage

This release uses a pinned CosyVoice checkout rather than a PyPI package:

git clone --recurse-submodules https://github.com/FunAudioLLM/CosyVoice.git
git -C CosyVoice checkout 074ca6dc9e80a2f424f1f74b48bdd7d3fea531cc
pip install -r CosyVoice/requirements.txt resemble-perth huggingface_hub

from huggingface_hub import hf_hub_download
import importlib.util, torchaudio

# the loader + text front-end ship in this repo
spec = importlib.util.spec_from_file_location(
    "amharic_tts", hf_hub_download("gabar-tech/cosyvoice2-amharic", "amharic_tts.py"))
amharic_tts = importlib.util.module_from_spec(spec); spec.loader.exec_module(amharic_tts)

tts = amharic_tts.load_amharic_tts(cosyvoice_repo="CosyVoice")  # downloads base (pinned) + fine-tune
wav = tts.generate(
    "ሰላም! ይህ ከጽሑፍ በቀጥታ የተፈጠረ የአማርኛ ድምፅ ነው። ዛሬ ነሐሴ 11 ቀን 2018 ዓ.ም. ነው።",
    audio_prompt_path="reference.wav",          # ~10 s of the voice to clone, with consent
    prompt_text="<exact transcript of reference.wav>")   # optional but much better
torchaudio.save("out.wav", wav, tts.sr)          # 24 kHz, PerTh-watermarked by the loader

Or from a checkout: python amharic_tts.py "ሰላም ዓለም።" --ref reference.wav --ref-text "<its transcript>" --out out.wav.

generate() normalizes the text, splits at sentence-final marks, merges very short chunks (they are less stable during synthesis), synthesizes each sentence against the reference and joins them. We ran the snippet above as written on NVIDIA A40, Ubuntu 22.04.5 LTS, python 3.11.10, torch 2.3.1+cu121, CosyVoice@074ca6dc (its requirements.txt) + resemble-perth, before publishing.

Watermarking

The included loader applies Resemble's PerTh watermark (resemble-perth, MIT) to every waveform generate() returns. The watermark is added by the loader rather than embedded in the weights, so it can be bypassed through another inference path and cannot be guaranteed for every use of the model. We ran the public detector over every demo clip from both models and the held-out eval outputs, with the natural reference recording as a negative control: present on all 116 checked files (detector confidence ≥ 0.5 on every generated file; the natural reference recording scores 0.0, so the detector is discriminating) (audit/watermark_verify.txt). If you build on this, keep the watermark in.

Intended use

• Amharic speech interfaces, audiobooks, education, accessibility, media production, with the consent of whoever's voice you clone.
• Research on low-resource TTS and Ethiopian language technology.

Out of scope

• Cloning someone's voice without their informed consent.
• Political persuasion, impersonating public figures, fraud (voice authentication included), harassment.
• Other languages. Use the original base checkpoint for those.
• Anything safety-critical or broadcast without a human listening first.

Limitations

• Gemination (consonant length; contrastive in Amharic, unwritten in normal spelling) isn't marked; the model reads minimal pairs from context and will get some wrong.
• The speech tokenizer is stock and frozen. Whatever detail it fails to capture about ejectives or gemination, no fine-tune of the stages behind it can recover.
• Numbers, abbreviations and Ge'ez numerals are expanded by amharic_text.py. Skip it and the model gets them raw.
• Zero-shot mode (reference transcript given) is noticeably more stable than cross-lingual mode (no transcript). If you have the transcript, pass it.
• Very short inputs can come out unstable; the loader merges short chunks, but single words are still best put in a sentence.
• The model inserts an "emm" or "uh" before some words. This is its main quality limitation. About 96% of the 193.63 hours of training audio is spontaneous image-description speech rather than read speech, and that portion runs about 21% slower by characters per second. This speaking style is a likely contributor; we have not isolated the cause.
• Expect conversational rather than neutral read-aloud delivery.
• Amharic only.

Risks and misuse

This model can clone a voice from roughly ten seconds of reference audio, so it can be misused for impersonation or fraud. Use it only with informed consent and human review.

Every voice in the training data was recorded under a licence that permits this use.

Attribution

• FunAudioLLM / Alibaba, for CosyVoice2 (Apache-2.0), the base model.
• Resemble AI, for the public PerTh watermarker (MIT).
• Digital Umuganda and the WaxalNLP contributors, Amharic speech data (CC-BY-SA-4.0).
• Mozilla Common Voice contributors, Amharic speech data (CC0-1.0).

Citation

@misc{gabar2026cosyvoice2amharic,
  title  = {CosyVoice2 Amharic: an Amharic fine-tune of CosyVoice2-0.5B},
  author = {{Gabar Technologies}},
  year   = {2026},
  url    = {https://huggingface.co/gabar-tech/cosyvoice2-amharic}
}