FarmerlineML/voxcpm2-akan-sft

VoxCPM2 Akan (Asante Twi), Full SFT

Full fine-tune of VoxCPM2 on Akan (Asante Twi) speech data. Trained by FarmerlineML for the Darli AI agricultural voice assistant.

Training Details

Parameter	Value
Base model	openbmb/VoxCPM2 (2B)
Method	Full SFT (all parameters)
Learning rate	1e-5
Batch size	2 (grad accum 16, effective batch 32)
Sample rate	16kHz (AudioVAE encoder input)
Final step	3028

Validation Loss:

Step	loss/total	loss/diff	loss/stop
0	0.899851	0.799370	0.066987
500	0.753233	0.704343	0.032593
1000	0.691477	0.670356	0.014081
1500	0.694497	0.682667	0.007886
2000	0.709386	0.686562	0.015216
2500	0.691271	0.673016	0.012170
3000	0.662822	0.647756	0.010043

Datasets:

TODO: confirm exact dataset repo names before publishing, the source manifests referenced FarmerlineML Akan/Twi TTS data but the exact repo IDs need verifying.

Usage

from voxcpm import VoxCPM
import soundfile as sf
import numpy as np

model = VoxCPM.from_pretrained(
    "FarmerlineML/voxcpm2-akan-sft",
    load_denoiser=False,
)

def trim_audio(wav, sr, silence_thresh=0.01, max_silence_secs=2.0):
    abs_wav = np.abs(wav)
    window  = int(0.05 * sr)
    n_wins  = len(abs_wav) // window
    max_sil = int(max_silence_secs / 0.05)
    silence_count, cut_sample = 0, len(wav)
    for w in range(n_wins):
        chunk = abs_wav[w * window:(w + 1) * window]
        if chunk.max() < silence_thresh:
            silence_count += 1
            if silence_count >= max_sil:
                cut_sample = (w - max_sil + 1) * window
                break
        else:
            silence_count = 0
    return wav[:min(cut_sample + int(0.1 * sr), len(wav))]

wav = model.generate(
    text="Ɛnyɛ asɔrekafoɔ no nkoaa na ɔyɛɛ biribi a ɛte saa",
    reference_wav_path="your_akan_speaker.wav",
    cfg_value=2.0,
    inference_timesteps=15,
    retry_badcase=False,
    max_len=max(50, len(text) * 4),
)
wav = trim_audio(wav, 48000)
sf.write("output.wav", wav, 48000)

Repo Structure

voxcpm2-akan-sft/
  model.safetensors        # Model weights (~9.2GB)
  audiovae.pth             # AudioVAE decoder
  config.json              # Model architecture config
  tokenizer.json           # Tokenizer
  training/
    train.log              # Full training log
    val_loss_summary.txt   # Validation losses per checkpoint
    training_state.json    # Final training state
  tensorboard/              # TensorBoard event files

Notes

• Reference audio is required at inference for voice identity anchoring
• Use max_len=max(50, len(text) * 4) to prevent hallucination after sentence end
• A post-generation 2-second silence trim is strongly recommended
• Trained on Asante Twi audio, use an Akan/Twi speaker reference clip for best results