Instructions to use bergetai/sten-lora with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- VoxCPM
How to use bergetai/sten-lora with VoxCPM:
import soundfile as sf from voxcpm import VoxCPM model = VoxCPM.from_pretrained("bergetai/sten-lora") wav = model.generate( text="VoxCPM is an innovative end-to-end TTS model from ModelBest, designed to generate highly expressive speech.", prompt_wav_path=None, # optional: path to a prompt speech for voice cloning prompt_text=None, # optional: reference text cfg_value=2.0, # LM guidance on LocDiT, higher for better adherence to the prompt, but maybe worse inference_timesteps=10, # LocDiT inference timesteps, higher for better result, lower for fast speed normalize=True, # enable external TN tool denoise=True, # enable external Denoise tool retry_badcase=True, # enable retrying mode for some bad cases (unstoppable) retry_badcase_max_times=3, # maximum retrying times retry_badcase_ratio_threshold=6.0, # maximum length restriction for bad case detection (simple but effective), it could be adjusted for slow pace speech ) sf.write("output.wav", wav, 16000) print("saved: output.wav") - Notebooks
- Google Colab
- Kaggle
VoxCPM2 — Swedish LoRA (Sturmark audiobook)
A LoRA adapter for openbmb/VoxCPM2 fine-tuned on a professionally narrated Swedish audiobook (~7.5 hours, 5,139 (clip, transcript) pairs, single speaker) to make the base model sound natural in Swedish — correct alveolar R, Swedish prosody and compound-word stress that the base model (trained mostly on other languages) does not produce.
The base model was evaluated against Zonos 3 Mini, KugelAudio, Qwen3-TTS and Piper on Swedish — none produced natural Swedish speech. This adapter closes that gap for one voice. Read the full evaluation story on the Berget AI blog.
Rights
The audiobook narration was contributed for ML training by the narrator without compensation (agreement on file with Berget AI, 2026-11). Do not use the adapter to generate speech intended to impersonate the narrator.
Usage
import json, torch, soundfile as sf
from huggingface_hub import snapshot_download
from voxcpm import VoxCPM
from voxcpm.model.voxcpm2 import LoRAConfig
ckpt = snapshot_download("bergetai/sten-lora")
# NOTE: lora_config.json is nested: {"base_model": ..., "lora_config": {...}}
raw = json.load(open(f"{ckpt}/lora_config.json"))
lora = LoRAConfig(**{k: v for k, v in raw["lora_config"].items()
if k in LoRAConfig.model_fields.keys()})
model = VoxCPM.from_pretrained(
"openbmb/VoxCPM2", load_denoiser=False,
device="mps", # cuda / cpu also work
optimize=False,
lora_config=lora,
lora_weights_path=ckpt,
)
wav = model.generate(
text="Hej! Det här är en svensk talsyntes tränad på öppen data.",
cfg_value=2.0, inference_timesteps=10)
sf.write("out.wav", wav, model.tts_model.sample_rate)
pip install voxcpm soundfile (Python 3.12, torch ≥ 2.5).
Training details
- Base: openbmb/VoxCPM2 (2B, tokenizer-free diffusion AR)
- Data: 5,139 segments (4,883 train / 256 val), 2–15 s each, 24 kHz source resampled to the AudioVAE encoder rate (16 kHz), transcripts from forced alignment via KBLab/kb-whisper-large
- Method: LoRA r=16, alpha=32, enable_lm + enable_dit, lr 1e-4, batch 1 × grad-accum 4, 4,000 steps (~3.3 epochs), fp32
- Hardware: 1× AMD MI300X (ROCm 7.2) — see ROCm notes below
ROCm notes (if training/serving on AMD)
- Install torch + torchaudio as a matched pair from the ROCm index:
pip install torch==2.11.0+rocm7.2 torchaudio==2.11.0+rocm7.2 --index-url https://download.pytorch.org/whl/rocm7.2 - Mixed bf16/fp32 crashes in forward pass — force fp32 (patch
_dtype()/ cast the model; amp autocast does not save you on ROCm here) train_voxcpm_finetune.pytakes--config_path(argbind YAML), not inline kwargs
Limitations
- Numbers are best spelled out in the source text ("fyrtiotre" rather than "43")
- Sentence-level line breaks in the source text produce natural pauses
- Long texts: raise
max_len(default 4096) or synthesize per sentence - The adapter carries the narrator's timbre and Swedish prosody — it does not make the base model multilingual
Model tree for bergetai/sten-lora
Base model
openbmb/VoxCPM2