MarkChen1214/Qwen3-ASR-1.7B-MLX-Mixed-4bit6bit8bit

Qwen3-ASR 1.7B MLX Mixed 4/6/8-bit

This is a size-optimized MLX conversion of Qwen/Qwen3-ASR-1.7B. It uses sensitivity-guided mixed-precision allocation to target the strongest measured quality point near 1.3 GB on Apple Silicon.

This model is not presented as an absolute quality winner. BF16 and the MLX Community 8-bit checkpoint retain better recognition quality. The value of this checkpoint is Q4-class quality at a substantially smaller weight size.

Quantization

• Affine quantization, group size 64
• Text decoder: 4-bit
• Audio tower: 8-bit by default
• Audio encoder layers 12–15: 6-bit
• Quantized modules: 197 at Q4, 24 at Q6, and 123 at Q8
• model.safetensors: 1.30 GB
• SHA-256: 751fd75aff6ab837528939139c37fa6573c772cedd6b5c6c05c5ab8d866a173b

The public allocation is recorded in quantization_recipe.json.

Matched evaluation

Apple M4 Pro, MLX 0.32.0, 100 samples per dataset, independent seed 2026. All models used the same mlx-audio runner, decoder settings, dataset rows, and normalization. Lower WER is better.

Model	Weights (GB)	LibriSpeech clean	LibriSpeech other	FLEURS Arabic	Macro WER
MLX BF16	4.08	1.31%	2.89%	9.24%	4.48%
MLX Community text-Q8	2.46	1.31%	2.78%	9.24%	4.44%
MLX Community text-Q4	1.60	1.44%	3.12%	12.10%	5.55%
External audio-Q8/text-Q4	1.31	1.48%	3.12%	11.99%	5.53%
This model	1.30	1.44%	3.17%	11.93%	5.51%

Compared with MLX Community text-Q4, this model is 19.09% smaller with a 0.04-point lower macro WER on this slice. Compared with the external audio-Q8/text-Q4 checkpoint, it is 0.96% smaller with practical quality parity (5.51% versus 5.53% macro WER).

These small WER differences should be interpreted as parity, not as evidence that quantization intrinsically improves the source model.

Full FLEURS Arabic

Model	Samples	WER	CER	RTFx
External audio-Q8/text-Q4	428	11.85%	4.75%	19.7x
This model	428	11.90%	4.69%	19.8x

The two compressed models are effectively tied on the full Arabic set. This checkpoint is intended for the general-ASR compression lane, not as an Arabic-specialized model.

Installation and loading

The tested environment used mlx-audio git commit 64e8416c303fb3b3463dab8eb4ebd78c55a87c1a:

pip install "mlx-audio[stt] @ git+https://github.com/Blaizzy/mlx-audio.git@64e8416c303fb3b3463dab8eb4ebd78c55a87c1a"
pip install huggingface-hub

The tested mlx-audio loader applies Qwen's broad audio exclusion before its saved per-layer quantization rules. This repository includes a small compatibility patch that makes explicit saved rules authoritative:

import sys
from huggingface_hub import snapshot_download

model_path = snapshot_download(
    "MarkChen1214/Qwen3-ASR-1.7B-MLX-Mixed-4bit6bit8bit"
)

sys.path.insert(0, model_path)
from mlx_audio_qwen_quant_patch import apply_patch

apply_patch()

from mlx_audio.stt import load

model = load(model_path, strict=True)
result = model.generate(
    audio="audio.wav",
    language="English",
    verbose=False,
)
print(result.text)

Without the compatibility patch, current mlx-audio versions may instantiate the explicitly quantized audio modules as BF16 and fail strict loading on the saved scale/bias tensors.

Reproducibility

The published weights were rebuilt from the pinned MLX BF16 revision and were byte-identical to the evaluated artifact.

• Experiment report
• Result manifest
• Conversion tool

Limitations

• BF16 and MLX Community Q8 are more accurate on the matched 100-sample gate.
• The 100-sample English results are sampled evaluations, not full-corpus LibriSpeech scores.
• Throughput is sensitive to machine temperature and run order; do not infer a speed advantage from small RTFx differences.
• Loading currently requires the included compatibility patch.

Credits and license

The base model is from the Qwen team. The unquantized MLX teacher is from mlx-community/Qwen3-ASR-1.7B-bf16. The model remains under the Apache 2.0 license of the upstream checkpoint.