tts-pl-matcha-v2 — Polish Matcha-TTS for a conversational robot

Non-autoregressive Polish TTS (Matcha-TTS, optimal-transport conditional flow matching, 20.9 M parameters) trained for a humanoid robot: low latency (2–4 ODE steps + HiFi-GAN), multi-speaker (8 target readers + blending), 8 style tokens for calm/neutral/lively delivery, and repaired question intonation. Trained from scratch (VCTK warm start) on consistency-filtered Wolne Lektury audiobooks plus YODAS and AZON conversational speech.

Files

path	what
runs_wcss/matcha_v2_final/checkpoints/final.ckpt	Lightning checkpoint (acoustic model, 22 speaker rows, 8 style rows) — for the playground / fine-tuning
export/matcha_v2_t2.onnx, export/matcha_v2_t4.onnx	acoustic model + HiFi-GAN in one ONNX graph, 2 / 4 ODE steps; inputs x (phoneme ids), x_lengths, scales=[temperature, length_scale], spks; output wav 22.05 kHz
export/matcha_v2_voices_t2.onnx, export/matcha_v2_voices_t4.onnx, export/voices.json	same graphs with 5 extra speaker rows baked in: 22 Kopa, 23 mix3 (0.5 Kopa + 0.3 Głogowski + 0.2 Faszczewska), 24 mix2 (Kopa + Głogowski), 25 Kopa calm (token 0), 26 Kopa lively (token 4) — the deployment recommendation from the GB10 benchmark
checkpoints/g_02500000	HiFi-GAN universal vocoder (for the checkpoint path)
data_v2/mix_v2_target_q/speaker_map.json	speaker id → reader / source; style_centroids.json → style token definitions
(external)	playground UI, CLI tools and the Matcha-TTS patch the checkpoint needs: https://github.com/machinekind/tts-pl-playground
samples/	synthesized test sentences (manifest.csv: file, voice, text)
RECIPE.md	full training procedure from scratch
ATTRIBUTION.md, LICENSE	data attribution (every book/reader/director) and CC BY-SA 4.0

Speakers and styles

Speaker ids 0–7 are the target readers (0 Bartosz Bielenia, 1 Katarzyna Faszczewska, 2 Wojciech Masiak, 3 Bartosz Głogowski, 4 Jan Staszczyk, 5 Marek Proszek, 6 Piotr Kopa, 7 Radosław Krzyżowski); ids 8–21 are base-training speakers kept for completeness. Blend voices by averaging embedding rows (synth_samples.py --voice "0.5*6+0.5*3" from tts-pl-playground).

Style token (4th input at training, an extra embedding summed with the speaker embedding): none / 5 = neutral, 0 and 6 = calm (pitch spread −1.2…−1.4 st), 1 / 4 / 7 = lively, 3 = question (rising terminal contour; use for yes/no and echo questions, not for wh-questions, which fall in Polish). Phonemization: espeak-ng pl via polish_cleaners (matcha_patch/ in tts-pl-playground), punctuation preserved — write real punctuation, "?" drives question intonation.

Quality (Whisper large-v3 WER, UTMOS, pitch spread)

Final model, 10 conversational test sentences per reader

voice	n	WER	CER	UTMOS	F0 spread [st]	chars/s	silence %
Bartosz Bielenia	10	0.035	0.013	3.34	1.85	10.6	18
Katarzyna Faszczewska	10	0.035	0.059	3.42	3.93	10.4	18
Wojciech Masiak	10	0.058	0.071	3.20	2.70	11.9	12
Bartosz Głogowski	10	0.047	0.061	3.36	2.77	11.1	12
Jan Staszczyk	10	0.070	0.071	3.08	3.22	10.6	13
Marek Proszek	10	0.035	0.059	3.19	2.50	10.9	14
Piotr Kopa	10	0.058	0.069	3.45	3.88	9.5	12
Radosław Krzyżowski	10	0.081	0.071	3.03	2.27	10.2	18

Stage-1 base model (before fine-tuning), same sentences

voice	n	WER	CER	UTMOS	F0 spread [st]	chars/s	silence %
Bartosz Bielenia	10	0.047	0.061	3.33	2.08	10.9	19
Katarzyna Faszczewska	10	0.093	0.078	3.37	2.96	10.9	16
Wojciech Masiak	10	0.047	0.061	3.30	2.63	12.4	13
Bartosz Głogowski	10	0.058	0.061	3.41	2.91	11.7	12
Jan Staszczyk	10	0.070	0.065	3.06	3.16	11.2	12
Marek Proszek	10	0.035	0.059	3.13	2.60	11.3	13
Piotr Kopa	10	0.058	0.063	3.45	2.61	10.3	11
Radosław Krzyżowski	10	0.058	0.065	2.97	3.29	10.9	17

Latency (Mac CPU, ONNX, 4 steps): ~0.4 s for a 4 s sentence; on an H100 / GB10-class GPU with CUDA graphs the target is < 100 ms to first audio. Piper control on the same readers: UTMOS 2.7–3.3, WER 7–22 % (see hvsr-robotics/tts-pl-piper-v2).

Training procedure (flattened)

See RECIPE.md for the complete, reproducible version. In short: ingest → per-clip UTMOS/DNSMOS/F0 statistics → prose-only Wolne Lektury readers ranked by consistency (low spread of quality and pitch), top 8 as targets → stage 1 base 40k steps on 22 speakers with style tokens (k-means on prosody statistics) → stage 2 fine-tune 12k steps on the 8 readers (text/audio-mismatch clips removed) → stage 3: relabel falling "?" as ".", oversample rising questions, token 3 = question → stage 4: short questions cut from the readers' own rising questions via the model's alignment, verified by ear. Adam 1e-4 → 5e-5, batch 64, bf16, one H100, ≈ 12 GPU-hours total.

Licence and attribution

• Weights: CC BY-SA 4.0 (LICENSE). The training audio is CC BY-SA 3.0 PL (Wolne Lektury), CC BY 3.0 (YODAS) and CC BY-SA 4.0 (AZON); ShareAlike propagates to the weights. Attribution for every book, reader, director and source is in ATTRIBUTION.md — keep that file with any redistribution or derivative.
• Warm start: Matcha-TTS matcha_vctk.ckpt (MIT; VCTK corpus CC BY 4.0, CSTR, University of Edinburgh). Vocoder: HiFi-GAN universal g_02500000 from the Matcha-TTS release (MIT). Code: Matcha-TTS (MIT), espeak-ng (GPL-3.0, runtime dependency).
• Voices are personal attributes. The CC licence covers the recordings, not the readers' personality rights. The recommended deployment is a blend of two or more readers under a neutral voice name; using a single reader's voice commercially should be cleared with the reader or Wolne Lektury. The model card names readers only as data sources.
• Synthetic speech should be disclosed as such where the listener could otherwise take it for a person (EU AI Act, art. 50).