twangodev/Ming-3.0-audio-tiny-NVFP4

Ming 3.0 Audio Tiny NVFP4

Audio understanding and reasoning, quantized for NVIDIA Blackwell.

GitHub · Ming project · Evaluation results · Methods

Model overview

Ming 3.0 Audio Tiny NVFP4 is a community quantization of inclusionAI's Ming 3.0 Audio Tiny, prepared with NVIDIA Model Optimizer. It accepts audio and text instructions for speech recognition, spoken question answering, audio understanding, and reasoning in English and Chinese.

The upstream model has 9.3B total parameters and 2.7B activated parameters per inference step. This release quantizes the routed experts in its Mixture-of-Experts Thinker. The original audio components, Talker, and AudioVAE are retained; the results below measure text responses, not synthesized speech.

Quantization

Property	Configuration
Format	NVFP4, 4-bit weights and activations (W4A4)
Method	Post-training quantization with MSE calibration
Tool	NVIDIA Model Optimizer 0.47.0
Quantized scope	Routed experts in 16 layers; 6,144 packed projections
Retained precision	Seven routed layers remain BF16; other components keep source precision
Block size	16
Validated hardware	NVIDIA RTX PRO 6000 Blackwell (SM120)

Selective precision was chosen after quality probing. The checkpoint contains both NVFP4 and BF16 routed experts; attention, shared experts, and audio components are outside the routed-expert quantization scope.

Performance

Measurement	BF16	NVFP4	Change
Thinker model allocation (GiB)	15.97	9.50	−40.5%
Text output tokens/s	7,533.84	7,775.16	+3.2%
Audio → text output tokens/s	8,991.92	8,981.39	−0.1%

Three-trial medians on one RTX PRO 6000 Blackwell, at client concurrency 512 for text and 1,024 for audio. Model allocation excludes KV cache, workspaces, CUDA graphs, and other allocations. These workloads show a substantial memory reduction and similar throughput; they do not establish a 4× end-to-end speedup.

Workload settings and measurements →

Evaluation

26,183 examples across eight full benchmark splits, completed on 2026-10-26. Values are percentages; lower WER/CER and higher accuracy are better. Local scores use the normalization and answer-parsing policies described below. Published scores are reference values, not matched-protocol baselines.

Benchmark	Metric	Local examples	Original (published)	BF16 (local)	NVFP4 (local)
FLEURS English	WER ↓	647	3.30	3.29	3.44
FLEURS Chinese‡	CER ↓	945	2.99	3.52	3.43
LibriSpeech test-clean	WER ↓	2,620	1.42	1.42	1.44
LibriSpeech test-other	WER ↓	2,939	3.06	3.04	3.07
MMAR	Accuracy ↑	1,000	67.70	63.90	65.20
MMAU test-mini†	Accuracy ↑	1,000	75.90	76.70	77.30
MMLU-Pro	Accuracy ↑	12,032	—	61.43	60.16
MMSU	Accuracy ↑	5,000	73.06	68.04	67.14

Evaluation methods, comparison details, and sources

The original unquantized scores are reported by inclusionAI. Our local Tiny BF16 and NVFP4 runs use the same full splits, prompts, and serving profile. “—” means the upstream images do not report a Tiny MMLU-Pro score.

All local examples were retained, with no request errors. These datasets overlap prior local development and checkpoint selection, so they are not untouched holdouts. The exact published evaluation recipe has not been established. † The upstream image labels its row “MMAU” without specifying the split; our result uses the public test-mini split.

‡ Chinese CER uses UltraEval-Audio text normalization and character-level scoring, applied after investigating the original strict scores. It handles number forms and some parenthetical text. UltraEval's default mixed character/Latin-word metric is not the CER shown here. The original strict scores were 8.54% BF16 and 8.45% NVFP4; original Tiny also scored 8.54% through independent Hugging Face inference.

Audio multiple-choice scoring resolves unambiguous option letters or text without consulting the gold answer during parsing. Capped or unparseable outputs count as wrong, with the full denominator retained. MMLU-Pro uses official answer extraction and counts unparseable outputs as wrong; capped outputs can still count if an answer is extracted. English ASR uses official Whisper normalization and corpus WER. These post-hoc scoring policies are documented separately from unchanged scorer outputs.

Generation was greedy with seed 42 and internal thinking disabled: 512 output tokens for ASR, 1,024 for audio multiple choice, and 4,000 for MMLU-Pro. Only ASR used 10-gram blocking. Separate budget/replay diagnostics did not replace any answers in this table.

These results supersede the historical 25,159-example subset for this card. Coverage, generation settings, and scoring changed; the checkpoint did not. The earlier results and their original methods remain available in the evaluation index.

HF leaderboard results link MMLU-Pro and LibriSpeech to their registered Hub tasks. The LibriSpeech entries rescore these same saved transcripts with the current Open ASR normalizer and compound-word matching; their values differ from the Whisper-normalized results in this table.

Full results and parity investigation · Machine-readable scores · Methods

Original-score sources: speech recognition, audio intelligence, text benchmarks.

Attribution and license

Base model by inclusionAI; quantization prepared by twangodev using NVIDIA Model Optimizer. This is an independent community release.

The base model is MIT licensed. Preserved code and assets retain their accompanying notices: MIT, Apache-2.0 code, fastText asset license, and inclusionAI's original notice.