MOSS-Transcribe-Diarize-0.9B-ONNX-INT8-ENC

ONNX export of MOSS-Transcribe-Diarize 0.9B with an INT8 audio encoder, intended for CPU inference.

Changes from the base model

Derived from soniqo/MOSS-Transcribe-Diarize-0.9B-ONNX-FP16, which is itself an ONNX export of OpenMOSS-Team/MOSS-Transcribe-Diarize. Two modifications:

• audio_encoder.onnx — weights converted FP16 to FP32, then MatMul weights dynamically quantized to per-channel INT8.
• decoder.onnx — weights converted FP16 to FP32, not quantized. Stored with external data in decoder.onnx.data. Both graphs must share one numeric type, so the decoder is FP32 rather than FP16.

No weights were retrained, and no architecture, tokenizer, or configuration was altered.

Measurements

Against the FP16 bundle on the ONNX Runtime CPU provider, 24 cores, 12 intra-op threads:

	FP16	this bundle
peak resident memory	5274 MB	3251 MB
20 s decode	4140 ms	3118 ms

Accuracy measured on FLEURS test, 50 clips each of ru_ru, en_us, it_it, cmn_hans_cn — WER for the first three, CER for Chinese. Of 200 clips, 186 produced identical transcripts, 9 improved and 5 regressed (two-sided sign test p = 0.42).

language	FP16	this bundle
en	5.76%	5.48%
it	5.80%	4.82%
ru	9.37%	8.80%
zh (CER)	9.99%	9.99%

Segment boundaries were compared separately over the same 199 clips. 127 produced byte-identical output including timestamps and activity markers, 186 produced the same number of turns, and where turns matched the boundary shift had a median of 0.00 s and a 90th percentile of 0.04 s. No turn changed whether it clears a two-second minimum, which is the threshold a downstream speaker-identity stage would care about. One clip assigned a different paragraph-local activity label.

FLEURS is single-speaker read speech, so none of this says whether speaker attribution changes on multi-speaker audio. That needs a diarization corpus and a DER comparison.

Files

audio_encoder.onnx, decoder.onnx, decoder.onnx.data, and the config, processor_config, preprocessor_config and vocab JSON files, matching the layout of the base repository.