MOSS-Transcribe-Diarize-0.9B-ONNX-FP16

ONNX conversion of OpenMOSS-Team/MOSS-Transcribe-Diarize, pinned to revision e6d68cdfcddbdad1a7e8454f0cb859cad76e2502. The model produces timestamped, speaker-attributed text in the form [start][Sxx]text[end].

Model

Property	Value
Parameters	908,513,280 total: 312,463,360 audio/VQ + 596,049,920 text decoder
Architecture	24-layer Whisper-medium encoder, 4x VQ adaptor, 28-layer Qwen3 decoder
Quantization	FP16 audio + FP16 decoder
Format	ONNX opset 18 with FP16 weights and dynamic cache axes
Bundle size	2.00 GiB
Audio	16 kHz mono; non-overlapping 30-second encoder chunks
Output	Timestamps, anonymous speaker labels, transcription text
Context	Dynamic host-owned K/V cache; the source supports 128k context, subject to host memory

Files

File	Size	Description
added_tokens.json	707 B	Additional token IDs
audio_encoder.onnx	596.2 MiB	FP16 Whisper encoder and VQ adaptor graph
chat_template.jinja	4.7 KiB	Prompt and audio-placeholder template
config.json	1.6 KiB	Root loader metadata and download-counting query file
decoder.onnx	1.40 GiB	Qwen3 decoder graph with documented K/V-cache inputs
export_config.json	4.7 KiB	Source revision, artifact hashes, and graph contract
generation_config.json	107 B	Greedy generation token defaults
merges.txt	1.6 MiB	Byte-pair merge rules
preprocessor_config.json	315 B	16 kHz Whisper feature-extractor settings
processing_moss_transcribe_diarize.py	10.7 KiB	Upstream processor implementation
processor_config.json	292 B	Audio-token and timestamp-marker settings
source_config.json	2.3 KiB	Pinned upstream model geometry
special_tokens_map.json	613 B	Special token definitions
tokenizer.json	10.9 MiB	Qwen tokenizer vocabulary and rules
tokenizer_config.json	503 B	Tokenizer configuration
validation.json	11.5 KiB	Measured quality, speed, memory, and parity results
vocab.json	2.6 MiB	Byte-pair vocabulary

Performance

Measured with greedy decoding on an Apple M5 Pro with 48 GB unified memory. Word error rate (WER), character error rate (CER), and real-time factor (RTF) are lower when better. Throughput is 1 / RTF, is higher when better, and reports how many seconds of audio are processed per wall-clock second. RTF excludes model loading. RSS is process memory sampled at clip boundaries; OS high-water RSS also includes transient peaks when the operating system reports it.

Slice	Samples	WER	CER	RTF	Throughput	Sampled / OS high-water RSS	Plain parity vs FP32
english	80	8.32	5.37	0.2798	3.6x real-time	7,421 / 7,497 MB	80/80

Aggregate inference phases across 80 clips: Processor 0.31 s, Audio encoder 90.74 s, Decoder prefill 12.84 s, Token decode 108.60 s, Other host work 0.01 s; 24.108 ms/generated token.

Paired runtime profile comparison

The same English subset was run in fresh processes for both rows.

Runtime profile	Samples	RTF	Throughput	Sampled / OS high-water RSS	Plain / raw parity
Previous FP16 profile	20	0.2551	3.92x real-time	7,343 / 7,409 MB	20/20 / 20/20
Recommended balanced profile	20	0.2540	3.94x real-time	7,200 / 7,267 MB	20/20 / 20/20

Multilingual precision check

Slice	Samples	WER	CER	RTF	Throughput	Plain parity vs FP32
german	10	13.37	5.99	0.2222	4.5x real-time	10/10
french	10	4.91	1.99	0.3542	2.8x real-time	10/10
msa	10	27.00	8.48	0.2789	3.6x real-time	10/10

Round trips

• 30-second overlapping AMI window: RTF 0.434; text=exact, speaker_sequence=exact, raw=exact. The source predicted 3 speakers.
• 33.12-second two-chunk German clip: RTF 0.228; text=exact, speaker_sequence=exact, raw=exact.

The multilingual check covers only ten German, ten French, and ten Modern Standard Arabic clips. It is a conversion check, not proof of the upstream model's full 50-language quality. The meeting round trip checks transcript and speaker-sequence parity but is not a diarization error-rate benchmark.

Usage

Python

from pathlib import Path

import onnxruntime as ort
from huggingface_hub import snapshot_download

bundle = Path(snapshot_download("soniqo/MOSS-Transcribe-Diarize-0.9B-ONNX-FP16"))
memory_info = ort.OrtMemoryInfo(
    "Cpu",
    ort.OrtAllocatorType.ORT_ARENA_ALLOCATOR,
    0,
    ort.OrtMemType.DEFAULT,
)
arena = ort.OrtArenaCfg(0, 1, -1, -1)  # same-as-requested growth
ort.create_and_register_allocator(memory_info, arena)

audio_options = ort.SessionOptions()
audio_options.add_session_config_entry("session.use_env_allocators", "1")
audio_options.add_session_config_entry("session.disable_prepacking", "1")
decoder_options = ort.SessionOptions()
decoder_options.add_session_config_entry("session.use_env_allocators", "1")
audio = ort.InferenceSession(
    str(bundle / "audio_encoder.onnx"), audio_options
)
decoder = ort.InferenceSession(
    str(bundle / "decoder.onnx"), decoder_options
)
print([value.name for value in decoder.get_inputs()])

The decoder accepts an empty cache for initial prefill and returns only newly generated K/V rows. The host appends those rows, performs greedy decoding, and parses [start][Sxx]text[end] output. Full signatures are in export_config.json.

Command line

hf download soniqo/MOSS-Transcribe-Diarize-0.9B-ONNX-FP16 --local-dir ./moss-transcribe-diarize

This downloads a complete low-level model bundle. SDK integration is tracked in speech-swift issue #388; until that integration lands, applications must implement the documented host contract around the exported graphs or weights.

Runtime contract

The audio encoder and decoder are separate graphs. The decoder supports empty-cache prefill and returns only new cache rows. Fixed Gather indices materialize GQA K/V heads while preserving the original query-head matrix-multiplication shape. The measured CPU profile shares one same-as-requested arena between both sessions while retaining graph optimization, memory patterns, and decoder prepacking. Audio prepacking is disabled to lower the resident working set. Prompt construction, audio chunking, generation, cache management, and transcript parsing remain host responsibilities. Machine-readable details and measured results are in config.json, export_config.json, and validation.json.

Source

• Upstream model
• Upstream implementation
• Technical report

No training was performed for this conversion. The source weights contain 908,513,280 parameters and are licensed under Apache 2.0.

Limitations

Speaker IDs are anonymous within each inference. The source model can emit malformed or overlapping timestamps and can miss speakers; conversion parity does not correct those behaviors. The measured CPU runtime used substantial memory; this is a compatibility export rather than the recommended Mac runtime. The validation here does not establish the upstream 90-minute claim for this deployment format.

Links

• speech-core — C++ runtime
• speech-android — Android SDK
• C++ docs — C++ runtime docs
• Android docs — Android setup docs
• soniqo.audio — website
• blog — blog

License

Apache License 2.0, inherited from the upstream model.