Nav8ha/fish-audio-drama-3-3bit

Drama 3 for MLX — mixed 3-bit (measured)

This checkpoint now lives at mlx-community/fish-audio-drama-3-3bit. The files are the same; use that repo for updates. Drama 3 support is merged into mlx-audio (#1018).

Built with Fish Audio.

An MLX conversion of Fish Audio Drama 3 for Apple Silicon: routed experts at 3 bits, dense projections and the audio tower at 8 bits. In our tests it matched 8-bit quality while being the smallest setting that kept quality — 18.5 GB on disk, about 22.5–24 GB peak while generating, so it fits a 32 GB Mac.

This is an unofficial community conversion. It is not released or endorsed by Fish Audio.

Quality, measured

20 voice-cloned English clips (10 sentences × 2 seeds), M5 Max. Whisper large-v3-turbo word error rate, ECAPA speaker similarity to a held-out clip of the reference speaker, UTMOS. "Failed" = WER above 20 % or no end of speech.

Experts / dense	Backbone	WER	Failed	Speaker sim	UTMOS
8 / 8	34.3 GB	1.6 %	0/20	0.625	3.92
4 / 8	19.3 GB	2.5 %	0/20	0.619	3.92
3 / 8 (this model)	16.9 GB	1.4 %	0/20	0.610	4.04
2 / 8	11.8 GB	0.9 %	0/20	0.624	3.84
3 / 3	14.1 GB	26.7 %	9/20	0.561	3.13

• The row for this model was measured on this exact checkpoint through mlx-audio. Other rows come from the same sweep in a standalone MLX port.
• In a blind listening test (6 lines × 3 versions), the listener did not prefer 8-bit over this model.
• 2-bit experts dropped the last two lines of a multi-speaker dialogue, so they are not offered here.
• Also checked on this setting: two-speaker dialogue, [sighs] / [laughs] cues, Hinglish, Hindi and a 43 s long-form script.

Usage

Drama 3 support is merged into mlx-audio (#1018). Until it is in a release, install from main:

pip install "mlx-audio[tts] @ git+https://github.com/Blaizzy/mlx-audio"

from mlx_audio.tts.utils import load_model

model = load_model("mlx-community/fish-audio-drama-3-3bit")
for result in model.generate(
    text="<|speaker:0|> So what did you ship this week? <|speaker:1|> [laughs] A spreadsheet.",
    ref_audio="reference.wav",
    ref_text="Transcript of the reference clip.",
):
    audio = result.audio  # 44.1 kHz

Speed: about 36 frames/s on an M5 Max (1.7× real time). Rebuild it yourself with:

python -m mlx_audio.tts.models.fish_drama3.convert --src drama-3 --dst drama-3-mlx-mixed-3bit \
    --expert-bits 3 --dense-bits 8 --tower-bits 8

License and use

Fish Audio Community License v1.0 (see LICENSE and NOTICE). Free for research and non-commercial use; commercial use only while annual revenue is under USD 500,000; hosted text-to-speech services need a separate license from Fish Audio. Do not clone a person's voice without their consent.