OsaurusAI/NemotronLabs-VoiceChat-11B-MXFP8

MXFP8 MLX bundle of nvidia/NVIDIA-NemotronLabs-VoiceChat-11B — uniform 8-bit reference — the quality baseline the JANG tiers are measured against.

VoiceChat is a full-duplex speech model: it listens on a continuous audio timeline, transcribes, answers in text, and synthesizes aligned speech — while handling interruptions, turn-taking, and tool calls.

Bundle

Field	Value
Size on disk	11.03 GiB
Source precision	fp32 (41 GiB) -> bf16 (21 GiB) -> this
Architecture	`nemotron_voicechat`
Backbone	`nemotron_h` hybrid — 56 layers (27 Mamba2 / 25 MLP / 4 attention)
Hidden size	4480
Audio in	16000 Hz mono
Audio out	22050 Hz, speaker `Aria`
Frame duration	0.08s (12.5 fps)
Perception	Conformer 24L d=1024, causal streaming
Speech codec	31x RVQ, codebook 1024
Shards	3
Bit distribution	{8: 563}

How it was quantized

Uniform 8-bit MXFP8 (group size 32). No calibration required; this is the quality reference.

Tensors deliberately kept at full precision

Tensor	Why
`tts_model.rvq_embs`	RVQ codebook — lookup entries, not a projection. Quantizing moves every centroid and corrupts all decoded audio.
`audio_prompt_latents.*`	Speaker identity (83 KiB). Also what custom voices are made of.
`mog_head.proj_mus`	read raw (reshaped, not called), so a packed weight breaks the mixture head
`embed_subword.embed_tokens`	its dtype is read to allocate a buffer; quantized it would be `uint32` and silently truncate floats

Modalities

Capability	Status
Speech in (ASR)	supported — RNN-T transducer, 16 kHz mono
Text out	supported
Speech out	supported — 22.05 kHz, built-in `Aria` voice
Tool calling	separate `function_head` channel — a full vocab-sized head, not text parsed after the fact
Full duplex	model supports interruptions / turn-taking; see Verification below for what has actually been exercised

Verification

Verified end-to-end on this bundle, offline turn from real audio:

loads and runs
ASR transcript correct and identical to the bf16 reference
coherent, on-topic reply text
speech output objectively speech-shaped — checked by RMS, zero-crossing rate and spectral centroid, not merely "a wav was produced" (silence and noise both produce wavs)
protected tensors verified byte-identical to the source

🚨 Not yet verified on this bundle: barge-in, turn-taking, and the tool channel under live duplex operation. Those require a streaming duplex session and have not been exercised here. Do not assume them.

Throughput is deliberately not quoted — it depends on host, runtime and settings, and a single number on a card is misleading.

Usage

Requires a runtime that implements nemotron_voicechat (upstream mlx-vlm >= 0.6.15).

from mlx_vlm.models.nemotron_voicechat import Model, ModelConfig, VoiceChatSession

# NOTE: a quantized bundle must be nn.quantize()d BEFORE load_weights,
# using the per-module map in config.json['quantization'] — otherwise
# packed uint32 codes are read as ordinary weights.

A worked loader, fixture prep and speech-comparison script are in the JANG examples.