Masterx/vibevoice-asr-bitnet-onnx

VibeVoice-ASR-BitNet — ONNX export

ONNX export of microsoft/VibeVoice-ASR-BitNet (dual ConvNeXt acoustic/semantic tokenizers over raw 24 kHz audio + QAT-ternary Qwen2.5-1.5B decoder), produced for WinSTT.

The decoder weights are the deployed BitNet ternary values: the per-tensor absmean ternarization from VibeASR.cpp's convert_lm_to_gguf.py (s = 1/mean(|W|); W' = round(clamp(W·s, −1, 1))/s) is applied to the q/k/v/o/gate/up/down projections at export, matching the official GGUF deployment bit-for-bit. Ternary×scale values are exactly representable in fp16 and in blockwise int4 (MatMulNBits, block 32, symmetric), so every tier decodes the same ternary model — tiers differ only in storage and kernels. Activations run in float (the ggml runtime quantizes activations to int8; float is equal or better).

Files

Tier	Graphs	Notes
fp32	audio_encoder.onnx (+.onnx_data), decoder_init.onnx, decoder_step.onnx, decoder_weights.data	parity reference (~10.4 GB)
fp16	*.fp16.onnx, decoder_weights.fp16.data	token-EXACT vs reference (~5.4 GB)
int4	*.int4.onnx, decoder_weights.int4.data	native BitNet tier, token-EXACT (~3.5 GB)

The int4 decoders re-encode the 196 ternary projection weights as com.microsoft::MatMulNBits (q = w/s + 8 ∈ {7,8,9}, block 32, per-block scale = the ternary scale — bit-exact packing, verified tensor-by-tensor at build and token-exact end-to-end); lm_head/embeddings stay fp16. There is intentionally no int8 tier: dynamic-activation int8 measurably degraded transcripts while being larger than the exact int4.

Shared across tiers: embed_tokens.bin (raw fp16 [151936 × 1536] row-major token-embedding table for host-side lookup), tokenizer.json (Qwen2.5), config.json.

Graph contract

audio_encoder(input_values[1,T] @24kHz) -> audio_features[1,N,1536]      # N = ceil(T/3200)
decoder_init(input_ids[1,S], position_ids[1,S],
             audio_features[1,N,1536], audio_offset[1])
    -> logits[1,1,V], present_keys/values [28,1,2,S,128]
decoder_step(input_embeds[1,1,1536], position_ids[1,1],
             past_keys/values [28,1,2,P,128])
    -> logits[1,1,V], present_keys/values [28,1,2,P+1,128]

decoder_init owns the embedding table and splices audio_features over the <|speech_pad|> (151648) positions starting at audio_offset. Prompt template (prompt_builder.h "text" format, no generation prompt — the model emits the <|im_start|>assistant turn itself):

<|im_start|>system\nYou are a helpful assistant that transcribes audio input into text output in JSON format.<|im_end|>\n
<|im_start|>user\n<|speech_start|>{<|speech_pad|> × N}<|speech_end|>\nThis is a {D:.2f} seconds audio, please transcribe it.<|im_end|>\n

With hotword context: \nThis is a {D:.2f} seconds audio, with extra info: {context}\n\nPlease transcribe it. Stop tokens: <|im_end|> (151645), <|endoftext|> (151643). Greedy decode.

Provenance

• Source checkpoint: microsoft/VibeVoice-ASR-BitNet fp32 QAT master safetensors.
• Conversion: official transformers convert_vibevoice_asr_to_hf.py (v5.14 key layout), ternarization per VibeASR.cpp, torch.onnx.export (TorchScript exporter, opset 17).
• Parity: validated tier-by-tier against the transformers reference implementation on 24 kHz speech samples (greedy token match).

License: MIT (inherited from the source model).