FireRedAudio-MLX-8bit

Apple Silicon MLX 8-bit quantized checkpoint for FireRedAudio (9B multimodal audio language model).

• 🐙 GitHub Repository & WebUI Studio: vanch007/mlx-FireRedAudio
• ⚡ Hardware Acceleration: Metal GPU native unified memory, fused SDPA attention, and Gated Delta Metal kernel.
• 🚀 Real-Time Factor (RTF):
  • ASR (Speech Recognition): RTF 0.1396 (~7.2x real-time speedup)
  • TTS (Voice Cloning): RTF 0.7529 (Real-time speech generation on Apple M3 Max)
• 💾 Memory Footprint: Only 13.54 GB active Metal memory (runs smoothly on 16GB / 24GB+ Mac devices).

---

Model Architecture & Quantization Details

• Backbone LLM: 9B Qwen3.5 language model with selective 8-bit affine quantization (group_size=64).
• Continuous Audio Autoencoder / Vocoder: RedAE VAE and Native ISTFT vocoder preserved in full FP32/BF16 precision for lossless audio phase fidelity.
• Audio Understanding Encoder: Whisper / Conv1D acoustic encoder preserved in high precision.
• Audio Generation Head: 11-layer Flow Matching DiT.

---

Quick Start

1. Installation

git clone https://github.com/vanch007/mlx-FireRedAudio.git
cd mlx-FireRedAudio
uv venv .venv
source .venv/bin/activate
uv pip install -e .

2. Download Model

hf download vanch007/FireRedAudio-MLX-8bit --local-dir models/FireRedAudio-8bit

3. Run Inference (CLI & Python)

from fireredaudio_mlx import FireRedAudioInference

engine = FireRedAudioInference(model_path="models/FireRedAudio-8bit")

# ASR Speech Transcription (RTF ~ 0.14)
res = engine.understand("assets/examples/asr_zh_fleurs.wav", task="asr")
print("Transcript:", res.answer)

# Zero-shot TTS Voice Cloning (RTF ~ 0.75)
audio_res = engine.tts(
    prompt_text="收到你的来信，我很高兴。",
    prompt_audio="assets/examples/tts_zh_prompt.wav",
    target_text="你好，欢迎体验 FireRedAudio 8-bit MLX 实时语音生成！",
)

4. Launch WebUI Studio

python run_webui.py --model models/FireRedAudio-8bit

Open your browser at http://127.0.0.1:7860.

---

Citation & References

• FireRedAudio Official: FireRedTeam/FireRedAudio
• Apple MLX: MLX Framework
• MLX Port & WebUI Studio: vanch007/mlx-FireRedAudio