Drama-3-MLX-8bit

This is the pure Apple Silicon MLX 8-bit affine quantized release of Fish Audio Drama 3.

Drama 3 is Fish Audio's state-of-the-art 35B parameter (3B active) Mixture-of-Experts (MoE) conversational text-to-speech model featuring a hybrid linear (GatedDeltaNet) + full softmax attention backbone and a 10-codebook Codebook-MTP audio decoder.

This 8-bit MLX release quantizes the 10,240 MoE experts using affine 8-bit quantization (group_size=64, bits=8), providing an optimal balance between fast memory bandwidth transfer and uncompromised acoustic nuance on Apple Silicon devices with 36GB+ Unified Memory.

Performance on Apple Silicon (M3 Max 128GB)

Precision	Model Size	Full Sentence Latency (7.06s Audio)	Step Latency	Real-Time Factor (RTF)	Whisper ASR Accuracy
4-bit	23.2 GB	24.38 s	160.4 ms/step	3.453	100.0%
8-bit (this model)	32.6 GB	28.29 s	210.0 ms/step	4.827	100.0%
BF16	69.0 GB	42.90 s	350.0 ms/step	5.238	100.0%

The 8-bit model runs 1.54x faster than BF16 on Apple Silicon unified memory due to reduced memory bandwidth traffic.

Features

Balanced Fidelity & Speed: Affine 8-bit MoE experts with full-precision DAC neural codec.
Zero-Shot Voice Cloning: High-fidelity cloning with reference audio and transcript.
Multi-Speaker Characters: Supports tags <|speaker:0|> to <|speaker:15|> (up to 16 distinct characters).
Point Tags & Emotion: In-text cues like [sighs], [laughs], [whispers].
44.1kHz High-Fidelity Audio: 10-codebook Modded DAC neural audio codec.

Quick Start

Installation

git clone https://github.com/vanch007/drama-3-mlx.git
cd drama-3-mlx
pip install -e .

CLI Synthesis

drama3-mlx tts   --model vanch007/Drama-3-MLX-8bit   --ref-audio ref.wav   --ref-text "Exact transcript of reference audio clip."   --text "你好，世界！这是 Fish Audio Drama 3 的 8-bit MLX 版本。"   --output outputs/sample.wav

Python API

from drama3_mlx.model import Drama3Model
import soundfile as sf

model = Drama3Model.from_pretrained("vanch007/Drama-3-MLX-8bit")

audio = model.tts(
    "Get the trust fund to the bank early, before the storm arrives tonight.",
    ref_audio="ref.wav",
    ref_text="Exact transcript of the reference audio clip."
)
sf.write("cloned_8bit.wav", audio, 44100)

References & Evaluation

GitHub Repository: https://github.com/vanch007/drama-3-mlx
Upstream Hugging Face: fishaudio/drama-3
Upstream GitHub: fishaudio/fish-speech
Evaluation Methodology: mlx-tts-evaluation

Downloads last month: -; Downloads are not tracked for this model. How to track