cguldogan/drama-3-mlx-q4

Superseded. Drama 3 is now in mlx-audio as fish_drama3 (#1018), with a different weight layout. This repo uses the layout of an earlier independent port and only loads with that fork. For a 4-bit checkpoint that loads with mlx-audio, use mlx-community/fish-audio-drama-3-4bit; for 8-bit experts with the rest in bf16, use mlx-community/fish-audio-drama-3-8bit. The original card is kept below for reference.

Drama 3 for MLX (q4)

Built with Fish Audio.

This is an MLX conversion of Fish Audio Drama 3 for Apple Silicon, with the routed experts quantized to 4-bit. Drama 3 is a 35B-parameter (3B active) mixture-of-experts text-to-speech model. It supports up to 16 speakers per script, voice cloning, inline cues such as [laughs] and [sighs], and editing by continuation.

This is a community conversion. It is not an official Fish Audio release, and Fish Audio does not endorse it.

Size	24 GB (sibling tier: drama-3-mlx-q8)
Peak memory while generating	~30 GB, so you need a Mac with 48 GB or more of unified memory (32 GB is too tight)
Speed on an M4 Pro (64 GB)	~31 frames/s in steady decode; real time is 21.5 frames/s
Audio	44.1 kHz mono

Usage

The model code lives on the drama3-mlx branch of cguldogan/mlx-audio. It is not in upstream mlx-audio yet.

pip install "mlx-audio[tts,server] @ git+https://github.com/cguldogan/mlx-audio@drama3-mlx"

python -m mlx_audio.tts.generate --model cguldogan/drama-3-mlx-q4 \
  --text "<|speaker:0|> [sighs] Fine. If we miss it, it's on you. [laughs] Well, maybe that's fair." \
  --ref_audio voice.wav --ref_text "Exact transcript of voice.wav." \
  --temperature 1.0 --top_p 0.9 --top_k 30

from mlx_audio.tts.utils import load_model

model = load_model("cguldogan/drama-3-mlx-q4")
for chunk in model.generate(text, ref_audio="voice.wav", ref_text="Transcript of voice.wav."):
    ...  # chunk.audio (44.1 kHz), chunk.text, chunk.codes

There is also a local test page with reference voices, a script editor, sampling controls and a list of takes:

python -m mlx_audio.tts.models.fish_omni.app --models cguldogan/drama-3-mlx-q4

Usage notes:

• Use the official sampling settings: temperature 1.0, top_p 0.9, top_k 30. The mlx-audio CLI's own defaults differ, so pass them explicitly.
• Pass reference audio as a file path. The model then resamples it to 44.1 kHz with a polyphase filter. Linear upsampling, which mlx_audio.utils.load_audio uses when it raises the sample rate, made generations ramble in testing.
• Lines heavy with cues miss on roughly 15–20% of seeds and run on past the text; re-roll the seed when that happens.

What was converted

• Quantized: the routed experts (4-bit, affine, group size 64) and the token embedding (8-bit). The original checkpoint stores the experts in block-FP8; they were dequantized and then re-quantized.
• Kept in bf16: the attention and linear-attention projections, shared experts, router, codebook tower and lm_head. The tower amplifies small errors from upstream: in parity tests, 8-bit versions of these layers made the codebook heads 5–30× worse than the production FP8 configuration.
• lm_head: trimmed to the rows the main token can be sampled from (eos plus the 4,096 semantic tokens).
• Codec: included as codec.safetensors, converted from the original codec.pth.

Evaluation

All measurements were taken on an M4 Pro with 64 GB.

• Parity: compared against the official PyTorch implementation on the first 4 backbone layers plus the full codebook tower, teacher-forced on real codec codes. Prompt tokens match exactly. Main-token logits match the production FP8 configuration. Codebook-head drift is at production level for 8-bit experts; with 4-bit experts, the first head drifts about 4× more.
• Codec: encodes bit-exactly the same as the original PyTorch codec.
• Intelligibility (Whisper large-v3-turbo): 8 cloned English sentences, every word correct on both tiers. A three-speaker dialog split into 3 chunks and an edit by continuation ("Wednesday" → "Thursday") were also verified.
• Turkish (q4): 8 sentences, cloned cross-lingually and with no reference, every word correct. Digits, decimals, percentages and times are read correctly; write years out in words if they read oddly.

License and use restrictions

These Fish Audio Materials are licensed under the Fish Audio Community License, Version 1.0. Copyright © 39 AI, INC. All Rights Reserved. See LICENSE and NOTICE. NOTICE describes how the materials were modified.

In short:

• Research and non-commercial use are free.
• Commercial use is free below USD 500k in annual revenue.
• Hosted TTS or voice-cloning services need a separate license.
• Do not clone a real person's voice without their consent, and do not use the model to impersonate anyone.
• Do not use its outputs to train other speech models.

Read the LICENSE file for the binding terms.