VITS Korean (mimic3 KSS) for Sentis (com.sky.sentis.vits)

Unity Sentis (com.unity.ai.inference 2.6.1) runner for the Korean VITS voice ko_KO-kss_low from mimic3-voices, as distributed by sherpa-onnx (vits-mimic3-ko_KO-kss_low). One fp32 graph, 22.05 kHz.

The upstream ONNX does not import into Sentis as-is (two If nodes). Both are a torch squeeze(1) on a dimension that is always 1, so they are folded to Squeeze(axes=[1]). The two RandomNormalLike nodes are replaced with slices of one fixed N(0,1) pool, so the same input always gives the same audio.

using Vits;
using Unity.InferenceEngine;

using var tts = new VitsTts("ko_kss", backend: BackendType.CPU);     // loads StreamingAssets/Vits
tts.Warmup();
var phrases = VitsPhrases.Load(Path.Combine(VitsTts.DefaultModelDirectory, VitsTts.PhrasesFile("ko_kss")));
float[] pcm = tts.Synthesize(phrases[0].Sentences);                  // 22.05 kHz mono

Install

Add the package by git URL, then run Tools ▸ VITS (KSS Korean) ▸ Install Models to StreamingAssets (or call Vits.Editor.VitsBaker.Bake()). Model~ is not included in player builds, so the files are copied to Assets/StreamingAssets/Vits.

https://huggingface.co/Sky-Kim/com.sky.sentis.vits

Text frontend (offline)

Ids come from mimic3's own frontend (espeak-ng phonemes → phonemes2ids with the voice's settings: # (id 5) between words, _ blanks, a final .), using Model~/mimic3_config/phonemes.txt. There is no in-Unity G2P. Model~/phrases_ko_kss.json holds precomputed ids for a test corpus, split into sentences; Tools~/build_phrases.py regenerates them.

Layout

Path	What
Model~/vits_ko_kss.sentis	fp32 VITS: input[1,T] int, input_lengths[1] int, scales[3] → output[1,1,S]
Model~/phrases_ko_kss.json	precomputed ids for the benchmark corpus
Model~/mimic3_config/	upstream voice config, phoneme table, README and LICENSE
Model~/fixtures/ko_kss/	ONNX Runtime reference tensors used by the tests
Runtime/	VitsTts (runner), VitsPhrases
Editor/VitsBaker.cs	copies Model~ into StreamingAssets
Tests/Editor/	Sentis-vs-ORT parity (CPU, GPUCompute) and end-to-end synthesis
Tools~/	surgery_bake_noise.py (If folding + noise baking), build_phrases.py, verify.py, asr_check.py

scales = [noise_scale 0.667, length_scale 1.0, noise_scale_w 0.8]; both noise scales still take effect after baking. Max 8192 frames (95 s) per call.

Verification (Sentis 2.6.1, Unity 6000.5.5f1, Apple Silicon)

Edge	Result
Original graph (noise as inputs) vs surgered graph, same pool slices	max|Δ| 0.0
ORT vs Sentis CPU	9.9e-5 (SNR 88 dB)
ORT vs Sentis GPUCompute	9.9e-5 (SNR 88 dB)
Back-transcription (Whisper small), 6-phrase corpus	CER 10.8 %, UTMOS 1.95

Benchmark (Apple M5 Max, standalone player, warm median)

	RTF (↓)	short-phrase TTFA
GPUCompute	0.007	33 ms
CPU	0.010	38 ms

Small VITS gains little from the GPU; CPU (Burst) is a good default, especially on mobile.

License

The voice weights are released under CC0 by the mimic3 project, but they are trained on the KSS dataset (CC BY-NC-SA 4.0), so treat this package as non-commercial. Package code: CC BY-NC-SA 4.0 together with the model. See LICENSE.md.