krapas-Regina 🌿 — Lithuanian TTS voice (OmniVoice fine-tune)

A Lithuanian text-to-speech voice: OmniVoice (k2-fsa, 0.6 B) fine-tuned on 3.05 hours of read Lithuanian speech from a single female speaker recorded for the LIEPA speech-synthesis project (Regina Jokubauskaitė). Runs on a Mac (MPS), an NVIDIA GPU, or CPU — nothing leaves the machine.

Listen first: sudi.lt/balsas — this checkpoint next to a second voice and to ElevenLabs v3 on the same sentences (homographs, nasal vowels, numbers, abbreviations, foreign names).

Vegetable-garden lineage of open Lithuanian speech models: svogūnas 🧅 → paprika 🌶️ (ASR) → bananas 🍌 (stress) → kmynas 🌱 (ASR) → krapas 🌿 (voice).

The part that matters for Lithuanian: stress marks in, not plain text

Lithuanian does not write stress, but stress decides pronunciation and meaning — kãsa (a braid) vs kasà (a till), mãno vs màno, sū́nus vs sūnùs. A model that sees only letters guesses, and guesses wrong often enough to be noticed.

So this model was trained on stress-marked text, and expects stress-marked text at inference. Two steps run before it:

1. Normalisation — digits, dates, times, units, abbreviations and initials become words in the right case. The model has never seen a digit and never will.
2. Stress marking — kristijonas/banano-kirtis-v2 (try it at bananas.sudi.lt) puts the marks on, in context.

Feed it unmarked text and it still speaks, but the stress becomes a lottery. Almost every stress error you hear is the stresser's, not the voice's — which is good news, because a wrong word in a dictionary is a one-line fix, not a retrain.

Usage

import torch
from omnivoice import OmniVoice, OmniVoiceGenerationConfig

m = OmniVoice.from_pretrained("kristijonas/krapas-Regina",
                              device_map="mps", dtype=torch.float32)   # "cuda" / "cpu" also fine
cfg = OmniVoiceGenerationConfig(guidance_scale=4, num_step=32)

wav = m.generate(text="Šiuõ metù galiù pažiūrė́ti pro lángą mán iš dešinė̃s mataũ spar̃ną.",
                 language="lt", generation_config=cfg, normalize_text=False)[0]

import soundfile as sf
sf.write("out.wav", wav, 24000)

Three settings that decide the result:

• guidance_scale=4 (the library default is 2). At 2 the model skips part of the stress marks; at 4 it follows the text. This is the single most important knob.
• num_step=32. 16 is roughly twice as fast with visibly more artefacts.
• normalize_text=False — the built-in normaliser does not know Lithuanian; do the normalising yourself before the stresser (see above).

Unicode form matters. Train and inference text must carry the same combining marks: NFC for the letters, the combining accents (U+0300 grave, U+0301 acute, U+0303 tilde) left as separate code points. If you feed fully-composed or fully-decomposed text, part of the marks stop being seen.

One sentence at a time. The training clips are short (mean 2.1 s, longest 5.1 s), so generate sentence by sentence and join with silence — 0.4 s between sentences, 0.9 s between paragraphs. A single request over ~15 s degrades.

Output is 24 kHz mono. On an M-series Mac a minute of speech takes about 1.3 minutes to generate; on an NVIDIA GPU it is faster than realtime.

Data

Source	meldynamics/liepa-tts — utterance-level audio recorded for the LIEPA speech-synthesis project, Vilnius University, CC BY 4.0. Not LIEPA-3, which is the ASR corpus
Speaker	Regina Jokubauskaitė, read speech, studio
Clips	5,124 (5,024 train / 100 dev), 3.05 h, mean 2.14 s, range 0.17–5.14 s
Text	stress-marked: 4,891 clips human-marked, 233 from an engine fallback
Punctuation	LIEPA transcripts carry none; a terminal . was appended to every line

Because the transcripts have no internal punctuation, the model learned sentence-final intonation from that appended period and little else. Questions and exclamations are weaker than in a voice recorded from punctuated script.

Training

{
  "init_from_checkpoint": "k2-fsa/OmniVoice",
  "learning_rate": 1e-5,
  "batch_tokens": 8192,
  "prompt_ratio_range": [0.0, 0.3],
  "steps": 5000,
  "save_steps": 500,
  "mixed_precision": "bf16",
  "weight_decay": 0.01,
  "warmup_ratio": 0.01,
  "seed": 42
}

One A100 80 GB, ~35 minutes including audio tokenisation. This is checkpoint 5000, picked by listening to 1000 / 2500 / 5000 — not by a metric. With 5 k clips there is no eval number that tells you which checkpoint sounds better; listen.

The repo bundles OmniVoice's audio_tokenizer/, so from_pretrained works on its own without the base snapshot.

What it does not do

• Digits, dates, units — the normaliser's job, not the model's.
• Foreign names it has not seen — it reads them as Lithuanian letters.
• Emotion — the data is calm read speech; that is what comes out.
• Whispering, singing, speaker mixing — no.
• Each generation differs slightly from the last; the same sentence twice is not the same waveform.

Ethics

This is a real person's voice, released as part of a corpus published under CC BY 4.0 for speech technology. Attribution to LIEPA / Vilnius University is required, and the voice should not be used to make that person appear to say things she did not say — endorsements, statements, or anything presented as a recording of her.

The same technology suits telephone fraud: a call in a familiar voice is no longer science fiction. Agree with your family on how you verify who is calling.

Licence

• Weights: CC BY-NC 4.0 — OmniVoice's weights are CC BY-NC, and this derivative inherits that. Personal and non-commercial use only.
• OmniVoice code: Apache 2.0.
• Training data: the LIEPA speech-synthesis recordings via meldynamics/liepa-tts, CC BY 4.0 (Vilnius University).
• A commercial Lithuanian voice needs a base with an Apache/MIT licence, trained from scratch on LIEPA — a different project.

Guide

The whole path — which sentences to record, how to record so the model does not learn your breathing, how to check and trim the takes, the training config, and how to wire up the stresser — is written up in Lithuanian: sudi.lt/gidai/savo-balsas.

Stressing the text before the voice model was Mindaugas Žvirblis's idea: 32 letters are too few for a voice, and marking stress turns them into ~80 distinct sounds.