Coda-saudi-final

VoxCPM 2 fine-tuned on Saudi dialectal Arabic for voice tone, naturalness, and dialect.

These are merged full weights — the LoRA adapter has been folded into the base weights, so this loads like any ordinary VoxCPM 2 checkpoint with no adapter code, no --lora-path, and no rank/alpha flags.

The unmerged adapters (all training steps) live at Rabe3/vox-loras.

Usage

This model is reference mode — pass a reference clip and the target text only. Do not pass the reference transcript as a prompt: VoxCPM 2's continuation mode concatenates prompt text onto the target text, which is not the format this model was trained on and will make it speak the prompt transcript first.

python -m voxcpm.cli clone \
  --model-path /path/to/Coda-saudi-final \
  --reference-audio reference/97Dxc4RjtnQ_0064.wav \
  --text "يا هلا والله، كيف حالك اليوم؟ إن شاء الله تكون بخير وعافية." \
  --seed 1234 --output out.wav

from voxcpm import VoxCPM

model = VoxCPM.from_pretrained("Rabe3/Coda-saudi-final", load_denoiser=False)
wav = model.generate(
    text="يا هلا والله، كيف حالك اليوم؟",
    reference_audio="reference/97Dxc4RjtnQ_0064.wav",
)

Reference audio

A known-good reference clip from the training speaker is bundled under reference/:

file	speaker	duration
reference/97Dxc4RjtnQ_0064.wav	97Dxc4RjtnQ_SPK1	5.60 s, mono 16 kHz

Its transcript is in reference/97Dxc4RjtnQ_0064.txt:

ما كان مأهول مرة، كانت بيوت متفرقة وهوا نظيف ومنطقة راقية

The transcript is supplied for provenance and for continuation-mode experiments. It is not needed for the reference-mode usage shown above.

Training

base model	openbmb/VoxCPM2
data	Rabe3/saudi-clean-vox — 2132 clips / 3.47 h, 7 speakers
split	2068 train / 64 val
method	LoRA, r=64, alpha=64, dropout=0.0
adapted modules	q_proj, k_proj, v_proj, o_proj on both the LM and the DiT
merged from	step 3000 (final)
optimizer	lr 1e-4, weight decay 0.01, 200 warmup steps, cosine schedule
batch	4 × 4 grad-accum = effective 16
steps	3000 (≈23 epochs), ~1 h on one H100 NVL

DiT adaptation was enabled deliberately: the DiT path carries timbre and prosody, which is what tone/naturalness transfer depends on.

Validation loss flattened from roughly step 1750 onward, oscillating inside a ±0.01 band — narrower than the noise of a 64-clip validation split. Best recorded validation diff loss was 0.8199 at step 2250; step 3000 sat at 0.8268. Checkpoint choice within that range is best made by listening, not by the loss number.

Merge details

LoRALinear computes y = xWᵀ + b + (alpha/r)·(xAᵀ)Bᵀ, so the exact merge is:

W ← W + (alpha/r) · (B @ A)

with alpha/r = 64/64 = 1.0. The update was accumulated in fp32 and cast back to bf16 to match the base checkpoint's storage format.

Verification performed on this artifact:

• All 192 adapted modules merged; 0 LoRA tensors unmatched.
• All 385 untouched tensors are bit-identical to the base checkpoint.
• Merged weights match the exact fp32 target to a mean relative error of 1.65e-3, below bf16 machine epsilon (3.9e-3) — i.e. the only deviation is storage rounding.
• Generating from the same reference, text, and seed, the merged model's output sits at a DTW-cosine distance of 0.0022 from base+adapter, versus 0.016 for either against the unmodified base — the merged model reproduces adapter behavior and is clearly distinct from base.

Because the model is autoregressive, bf16 rounding can still shift the stop decision by a frame or two, so generated durations may differ slightly from the adapter setup (4.32 s vs 4.16 s on the sample above). Content and voice are the same.

Limitations

• Trained on 3.47 h from 7 speakers, one per source video, with heavily uneven counts (556 / 479 / 443 / 235 / 138 / 128 / 89 clips).
• Only 84 distinct clips ever served as references during training, each reused ~25×. Robustness to unseen reference prompts is correspondingly less well covered than the clip count suggests.
• Training text is unvocalized Arabic transcript.
• Train loss/stop collapsed to ~1e-5 while validation held near 0.044, indicating some memorization of the training set.
• The bundled reference clip is a training clip, so demos using it flatter the model slightly. Use a validation clip for an honest read.

License

The base model is Apache-2.0, but the training data (Rabe3/saudi-clean-vox) is CC-BY-NC-4.0. These derived weights are released under CC-BY-NC-4.0 — non-commercial use only.