OsaurusAI/NemotronLabs-VoiceChat-11B-JANG_4

JANG_4 MLX bundle of nvidia/NVIDIA-NemotronLabs-VoiceChat-11B — the recommended default — Hessian-allocated 4-bit with AWQ + imatrix.

VoiceChat is a full-duplex speech model: it listens on a continuous audio timeline, transcribes, answers in text, and synthesizes aligned speech — while handling interruptions, turn-taking, and tool calls.

Bundle

Field	Value
Size on disk	7.92 GiB
Source precision	fp32 (41 GiB) -> bf16 (21 GiB) -> this
Architecture	`nemotron_voicechat`
Backbone	`nemotron_h` hybrid — 56 layers (27 Mamba2 / 25 MLP / 4 attention)
Hidden size	4480
Audio in	16000 Hz mono
Audio out	22050 Hz, speaker `Aria`
Frame duration	0.08s (12.5 fps)
Perception	Conformer 24L d=1024, causal streaming
Speech codec	31x RVQ, codebook 1024
Shards	2
Bit distribution	{4: 491, 5: 42, 6: 19, 8: 11}

How it was quantized

Three calibration methods, all driven by one capture pass over real conversational audio (562 modules, 3.38 M row-samples, exercising speech-in AND speech-out):

Method	Detail
Hessian-trace allocation	bits assigned by measured `tr(H)·‖W‖²_F` per module, not by tensor name
AWQ	alpha=0.25, salient-channel scaling absorbed into the producing RMSNorm across 112 norm groups / 204 projections
imatrix refit	activation-weighted affine fit replacing RTN — mean weighted rel-err 0.0488

Measured sensitivity drove the floors — the TTS subword encoder turned out to be the most sensitive component in the whole model (8x the mean trace of lm_head), and lm_head / function_head measured identically, so they get equal width.

Tensors deliberately kept at full precision

Tensor	Why
`tts_model.rvq_embs`	RVQ codebook — lookup entries, not a projection. Quantizing moves every centroid and corrupts all decoded audio.
`audio_prompt_latents.*`	Speaker identity (83 KiB). Also what custom voices are made of.
`mog_head.proj_mus`	read raw (reshaped, not called), so a packed weight breaks the mixture head
`embed_subword.embed_tokens`	its dtype is read to allocate a buffer; quantized it would be `uint32` and silently truncate floats

Modalities

Capability	Status
Speech in (ASR)	supported — RNN-T transducer, 16 kHz mono
Text out	supported
Speech out	supported — 22.05 kHz, built-in `Aria` voice
Tool calling	separate `function_head` channel — a full vocab-sized head, not text parsed after the fact
Full duplex	model supports interruptions / turn-taking; see Verification below for what has actually been exercised

Verification

Verified end-to-end on this bundle, offline turn from real audio:

loads and runs
ASR transcript correct and identical to the bf16 reference
coherent, on-topic reply text
speech output objectively speech-shaped — checked by RMS, zero-crossing rate and spectral centroid, not merely "a wav was produced" (silence and noise both produce wavs)
protected tensors verified byte-identical to the source

🚨 Not yet verified on this bundle: barge-in, turn-taking, and the tool channel under live duplex operation. Those require a streaming duplex session and have not been exercised here. Do not assume them.

Throughput is deliberately not quoted — it depends on host, runtime and settings, and a single number on a card is misleading.

Usage

Requires a runtime that implements nemotron_voicechat (upstream mlx-vlm >= 0.6.15).

from mlx_vlm.models.nemotron_voicechat import Model, ModelConfig, VoiceChatSession

# NOTE: a quantized bundle must be nn.quantize()d BEFORE load_weights,
# using the per-module map in config.json['quantization'] — otherwise
# packed uint32 codes are read as ordinary weights.

A worked loader, fixture prep and speech-comparison script are in the JANG examples.