Matcha-TTS + HiFi-GAN v1 for Sentis (com.sky.sentis.matcha.v1)

Unity Sentis (com.unity.ai.inference 2.6.1) port of Matcha-TTS (OT-CFM flow matching, LJSpeech English) with the original HiFi-GAN v1 vocoder. It runs fp32 and unquantised, exported from the author's PyTorch checkpoints (matcha_ljspeech.ckpt, generator_v1). The Euler ODE loop is driven from C#, so the step count is a runtime knob.

For a much smaller and faster vocoder with the same acoustic model, see com.sky.sentis.matcha.v2.

using MatchaV1;
using Unity.InferenceEngine;

using var tts = new MatchaTts(backend: BackendType.GPUCompute);   // loads StreamingAssets/MatchaV1
tts.Steps = 5;                                                      // 2 = fastest, 5 = default
tts.Warmup();

var phrases = MatchaPhrases.Load(Path.Combine(MatchaTts.DefaultModelDirectory, "phrases_en.json"));
float[] pcm = tts.Synthesize(phrases[0].Sentences);                // 22.05 kHz mono
var clip = AudioClip.Create("tts", pcm.Length, 1, MatchaTts.SampleRate, false);
clip.SetData(pcm, 0);

Install

Add the package by git URL, then run Tools ▸ Matcha-TTS v1 ▸ Install Models to StreamingAssets (or call MatchaV1.Editor.MatchaBaker.Bake()). Model~ is hidden from the asset pipeline and not included in player builds, so the .sentis files are copied to Assets/StreamingAssets/MatchaV1.

https://huggingface.co/Sky-Kim/com.sky.sentis.matcha.v1

Text frontend (offline)

The graphs take phoneme ids: english_cleaners2 → espeak-ng en-us IPA (with stress and punctuation) → Matcha symbol table → a blank (id 0) interspersed between symbols. There is no in-Unity G2P. Model~/phrases_en.json holds precomputed ids for a test corpus; produce ids for your own text with Tools~/make_fixture.py (see Tools~/common.py: text_to_ids).

Layout

Path	What
Model~/matcha_encoder.sentis	text encoder + duration predictor: x[1,T] → mu_x[1,80,T], logw[1,1,T]
Model~/matcha_estimator.sentis	one Euler step of the CFM decoder: x, mask, mu [1,80,L], t[1] → v[1,80,L]
Model~/hifigan_v1.sentis	HiFi-GAN v1: mel_norm[1,80,F] → wav[1,F·256] (denormalisation and clamp inside)
Model~/phrases_en.json	precomputed ids for the benchmark corpus, split into sentences
Model~/fixtures/	ONNX Runtime / PyTorch reference tensors used by the tests
Runtime/	MatchaTts (runner), MatchaPhrases
Editor/MatchaBaker.cs	copies Model~ into StreamingAssets
Tests/Editor/	Sentis-vs-ORT parity per graph (CPU, GPUCompute) and end-to-end synthesis
Tools~/	export (export.py), fixtures/phrases (make_fixture.py), ORT reference chain

Pipeline

1. Encoder on the ids.
2. Length regulation on the host (MatchaTts.Expand): w = ceil(exp(logw)) · length_scale, y_len = max(1, floor(Σw)), L = ceil(y_len/4)·4; frame j copies encoder column i where cum[i-1] ≤ j < cum[i].
3. x = N(0,1) · temperature (seeded), then for k = 0..N-1: v = estimator(x, mask, mu_y, k/N), x += v/N. The update runs in a compiled Functional graph, so x stays on the device.
4. x[:, :, :y_len] → HiFi-GAN v1.

Defaults: temperature 0.667, length_scale 1.0, N = 5. The optional HiFi-GAN denoiser of the Matcha CLI is not included.

Verification (Sentis 2.6.1, Unity 6000.5.5f1, Apple Silicon)

Edge	Result
PyTorch (original) vs ONNX Runtime	mu_x 2.3e-6, logw 4.6e-6, mel 4.9e-6 (N=5), waveform 4.5e-5
ORT vs Sentis CPU	encoder ≤ 4.3e-6, estimator 5.1e-6, vocoder 6.3e-6 (waveform SNR 107 dB)
ORT vs Sentis GPUCompute	encoder ≤ 4.3e-6, estimator 5.7e-6, vocoder 6.2e-6 (waveform SNR 104 dB)
Back-transcription (Whisper small)	WER 6.0 % over the 5-phrase corpus; UTMOS 3.67

Benchmark (Apple M5 Max, standalone player, warm median)

	RTF (↓)	short-phrase TTFA
GPUCompute, 5 steps	0.015	55 ms
GPUCompute, 2 steps	0.013	41 ms
CPU, 5 steps	0.065	170 ms

HiFi-GAN v1 dominates the cost, so the step count matters little here. With HiFi-GAN v2 (com.sky.sentis.matcha.v2) the GPU RTF drops to 0.006. Measure on your own target device.

License

Matcha-TTS code and LJSpeech checkpoint © Shivam Mehta et al., MIT. HiFi-GAN © Jungil Kong (jik876), MIT. LJSpeech is public domain. See LICENSE.md.