litert-community/SenseVoiceSmall

SenseVoice-Small — fixed-shape LiteRT (TFLite) buckets, int8

FunAudioLLM/SenseVoiceSmall (~234 M params, non-autoregressive SAN-M encoder + CTC) converted to LiteRT for on-device use, unchanged weights. One forward pass per utterance — no decoder loop.

What is different from the upstream ONNX

• Static shapes, one graph per window (3 / 5 / 10 / 30 s). A dynamic-shape graph keeps the XNNPACK delegate from attaching on LiteRT; these graphs run 10× (fp32) to 25× (i8) faster with XNNPACK than on the built-in kernels.
• Padding is masked, not heard. speech_lengths stays a graph input: pad the features of a shorter clip with zeros up to the bucket and feed the true length. Measured: padded + true length is bit-identical to the unpadded graph (max |Δlogit| 0.0000). Feeding padded audio without the length instead wrecks it (real-command CER 0.009 → 0.44 at a 5 s pad).
• Weights int8. w8 = weight_only_wi8_afp32 (activations fp32), i8 = dynamic_wi8_afp32 (activations quantized per call — faster, slightly worse in noise).

Files

File	Size	Window	speech input	Note
sensevoice_small_3s_w8.tflite / _i8	231 MB	3 s	[1, 560, 50]	short commands
sensevoice_small_5s_w8.tflite / _i8	231 MB	5 s	[1, 560, 84]	voice commands, recommended
sensevoice_small_10s_w8.tflite / _i8	231 MB	10 s	[1, 560, 167]	sentences
sensevoice_small_30s_w8.tflite / _i8	231 MB	30 s	[1, 560, 500]	dictation
am.mvn	11 KB			CMVN (AddShift + Rescale, 560 each)
chn_jpn_yue_eng_ko_spectok.bpe.model	0.4 MB			SentencePiece, 25055 pieces
tokens.txt	0.3 MB			id<TAB>piece, same vocab, for runtimes without SentencePiece
config.yaml				upstream config (front-end parameters)
SHA256SUMS

Load only the bucket(s) you need — every graph carries the full weights.

REQUIRED integration notes (tensor contract)

Bind inputs by name (signature serving_default; the converter orders them alphabetically).

Tensor	Shape	Type	Value
speech	[1, 560, N]	f32	LFR+CMVN features, feature-major (transpose of the usual [T, 560]), zero-padded to N
speech_lengths	[1]	i32	true number of LFR frames (≤ N)
language	[1]	i32	query id: auto 0, zh 3, en 4, yue 7, ja 11, ko 12, nospeech 13
textnorm	[1]	i32	withitn 14 (punctuation + digits), woitn 15
ctc_logits (out)	[1, N+4, 25055]	f32	keep the first speech_lengths + 4 rows
encoder_out_lens (out)	[1]	i32	= speech_lengths + 4

Front end (must match exactly — this is funasr WavFrontend):

1. 16 kHz mono, waveform × 32768.
2. Kaldi fbank (torchaudio.compliance.kaldi.fbank): 80 mel bins, 25 ms window / 10 ms shift, Hamming, snip_edges=True, energy_floor=0, dither 0 (funasr defaults to 1.0, which makes runs non-deterministic); Kaldi defaults otherwise (pre-emphasis 0.97, DC removal).
3. LFR m=7, n=6: prepend 3 copies of the first frame, stack 7 frames every 6, pad the tail with the last frame (→ 560-dim, 60 ms per frame, ceil(T/6) frames).
4. CMVN from am.mvn: (x + shift) * scale.

Decode: greedy CTC — argmax per row, collapse repeats, drop id 0 (blank), map ids through tokens.txt (▁ = space). The first four emitted tokens are tags (<|ko|> <|NEUTRAL|> <|Speech|> <|woitn|>); strip <|…|>.

⚠️ Evaluation caveats — read before using the numbers

• gate40 is the 40-clip TTS-synthesized Korean short-command holdout used to rank this project's Whisper ACFT tiers. It is valid for ranking only (Spearman ρ = 1.00 against real recordings) and runs ~2.8× harsher than real audio in absolute CER.
• References average 3.6 characters: one wrong syllable is CER 0.25–0.33.
• Zeroth-Korean test sentences also appear in Zeroth train (456 / 457), and SenseVoice's training data is undisclosed — treat the Zeroth number as optimistic.
• CER = Levenshtein on the string with spaces removed and punctuation stripped (what a command matcher sees), woitn, language forced to ko.

Validation (desktop, Windows x86-64 CPU, LiteRT 2.1.6, XNNPACK, 4 threads)

Set	n	PyTorch (funasr, CUDA)	tflite fp32	w8	i8
gate40 (TTS, ranking only)	40	0.169 · 21 exact	0.204 · 20	0.204 · 20	0.204 · 20
real commands, quiet room	13	0.009 · 11	0.006 · 12	0.006 · 12	0.006 · 12
real commands, field noise	9	0.296 · 4	0.269 · 5	0.269 · 5	0.333 · 5
Zeroth-Korean test (see caveat)	457	0.024	0.023	0.023	0.024
FLEURS ko dev	64	0.071	0.068	0.068	0.068

• Conversion parity: fp32 tflite vs onnxruntime on the same features — max |Δlogit| ≤ 3.6e-4 on commands, 0.029 on 20 s sentences, CTC argmax agreement 100 %, transcripts identical on all 583 clips. The PyTorch column differs only by CUDA-vs-CPU numerics flipping borderline tokens (e.g. 확대 → kana in both, on different clips).
• Against Whisper ACFT-KO on the same gate, same protocol (first 24 clips, punctuation kept): SenseVoice-Small 0.198 (PyTorch) / 0.247 (tflite) vs turbo-acft 0.182, medium 0.208, base 0.305, tiny 0.457 — between turbo and base, at a quarter of turbo's size.

Invoke latency per window (model only, p50):

Window	fp32	w8	i8
3 s	128 ms	134 ms	98 ms
5 s	145 ms	172 ms	121 ms
10 s	290 ms	286 ms	211 ms
30 s	827 ms	815 ms	691 ms

Not yet measured on an Android device.

Caveats

• Stock weights. Two Korean fine-tunes were measured and lost to stock on every set (an EDIE fine-tune: Zeroth CER 0.48; an in-house Zeroth fine-tune: real-command CER 0.020 vs 0.009).
• Field noise is the weak spot (fan/wind/vehicle recordings: 5 / 9 exact).
• Language forcing is soft: a few 1-syllable clips come out in kana even with ko.

License / attribution

Weights: FunAudioLLM SenseVoiceSmall, under the FunASR model license. Conversion only — no weights were changed. Paper: FunAudioLLM, arXiv 2407.04051.