carloshuang1224/mega-asr-mlx

Mega-ASR MLX

A high-accuracy automatic speech recognition model optimized for Apple Silicon (M1/M2/M3/M4) using MLX.

This model is an MLX port of Qwen3-ASR-1.7B with Mega-ASR LoRA adapters pre-merged, plus an Audio Quality Router for degraded audio detection.

Model Architecture

Component	Details
Audio Encoder	3-layer Conv2D stem + 24-layer Transformer (d_model=1024, 16 heads)
Decoder	Qwen3 1.7B — 28 layers, hidden=2048, 16 GQA heads (8 KV)
Aligner	conv_out (7680→1024) + proj1 (1024→1024) + proj2 (1024→2048)
Router	Mini Transformer (d_model=256, 4 layers, 4 heads) — binary audio quality classifier

Total parameters: ~1.7B (decoder) + ~350M (encoder)

Quick Start

Install

pip install mega-asr-mlx

Download the model

huggingface-cli download carloshuang1224/mega-asr-mlx --local-dir ./mega-asr-mlx

Inference

CLI:

mega-asr --audio speech.wav --language English

Python API:

from mega_asr_mlx import MegaASRMLX

model = MegaASRMLX("./mega-asr-mlx")
text = model.transcribe("speech.wav", language="English")
print(text)

Advanced Usage

from mega_asr_mlx import MegaASRMLX

model = MegaASRMLX(
    "./mega-asr-mlx",
    use_lora=True,           # always use LoRA (pre-merged)
    router_threshold=0.5,    # audio quality routing threshold
    max_new_tokens=256,      # max generated tokens
)

# Transcribe with routing info
result = model.transcribe("speech.wav", return_route_info=True)
# {"text": "...", "use_lora": True, "degraded_prob": 0.12}

# Transcribe numpy array
import soundfile as sf
audio, sr = sf.read("speech.wav")
text = model.transcribe(audio, sr=sr)

Performance

On Apple Silicon (M-series), with KV cache optimization:

• ~50 tokens/second generation speed
• Decoder weights: 3.8 GB (float16)
• Encoder weights: 606 MB (float16)
• Router weights: 2.3 MB (float16)

Files

mega-asr-mlx/
├── decoder.safetensors      # Qwen3 decoder weights (3.8 GB)
├── encoder.safetensors      # Audio encoder weights (606 MB)
├── router.safetensors       # Audio quality router (2.3 MB)
├── config.json              # Model configuration
├── decoder_config.json      # Decoder architecture config
├── router_config.json       # Router architecture config
├── preprocessor_config.json # Audio preprocessing (Whisper-style mel)
├── generation_config.json   # Generation defaults
├── tokenizer_config.json    # Qwen2 tokenizer config
├── vocab.json               # Token vocabulary
├── merges.txt               # BPE merges
└── chat_template.json       # Qwen3-ASR chat template

Source Code

The full inference pipeline is bundled in the mega-asr-mlx pip package. After installing it, the source is available in the mega_asr_mlx/ directory.

Key files:

• src/inference.py — End-to-end transcription script
• src/model.py — MegaASRMLX class with full pipeline
• src/audio_encoder.py — MLX audio encoder implementation
• src/router.py — Audio quality router
• src/convert.py — Weight conversion from PyTorch to MLX

How It Works

1. Audio Input → 16kHz mono waveform
2. Mel Extraction → 128-bin log-mel spectrogram (Whisper-style)
3. Audio Encoder → Conv2D stem downsamples 8×, then 24 Transformer layers encode
4. Decoder Input → Template: <|im_start|>user\n<|audio_start|> [encoder_outputs] <|audio_end|><|im_end|>\n<|im_start|>assistant\n
5. Generation → Qwen3 decoder auto-regressively generates text with KV cache

Credits

• Base model: Qwen/Qwen3-ASR-1.7B by Alibaba Qwen Team
• LoRA adapter: Mega-ASR by VoiceInk
• MLX port: Converted and optimized for Apple Silicon

License

Apache 2.0