Zyphra/ZONOS3-Mini-hf

ZONOS3-Mini for 🤗 Transformers

ZONOS3-Mini is Zyphra's compact multilingual text-to-speech model. This repo packages it for plain transformers (remote code, trust_remote_code=True): the language model, the causal multi-resolution iSTFT codec decoder (keep2_plain) and the MECT-B2 speaker encoder in one checkpoint, with the SuperBPE tokenizer and the 27 release voices. It synthesizes 44.1 kHz mono speech on CPU or GPU with nothing but torch and transformers.

Model details, samples and evaluations: the blog post. The native runtimes (CUDA, MLX, ggml, WebGPU) and the full HTTP API live in the ZONOS3 repository, which also holds the converter that builds this repo (tools/convert_hf.py).

Usage

pip install "transformers>=4.57" torch soundfile
pip install zyphra-textnorm   # optional, recommended: reads numbers, dates, currencies and units aloud

import soundfile as sf
import torch
from transformers import AutoModel

model = AutoModel.from_pretrained("Zyphra/ZONOS3-Mini-hf", trust_remote_code=True)
model = model.to("cuda" if torch.cuda.is_available() else "cpu")

out = model.generate("Hello! This is ZONOS3-Mini running in transformers.", voice="Savannah", seed=0)
sf.write("hello.wav", out.waveform.numpy(), out.sampling_rate)

AutoModelForTextToWaveform loads the same class. The checkpoint's LM is bfloat16 (the release dtype): transformers 5 loads it as stored, transformers 4.x in float32 unless you pass dtype=torch.bfloat16. The codec decoder and the speaker encoder always stay in float32. dtype=torch.float32 gives the reference numerics on CPU.

Voices

Pick one voice source per call:

Argument	Conditioning
voice="Savannah"	a release voice (below): its reference codes, or its speaker embedding with speaker_mode="embedding"
speaker_audio=wav, speaker_sample_rate=sr	a clip ([samples] or [channels, samples], any rate) through the speaker encoder
speaker_embedding=emb	a 192-d speaker embedding (model.embed_speaker(wav, sr) computes one)
speaker_reference_codes=codes	precomputed codec codes [frames, 10] of a clip (at most 60 s used)

speaker_mode defaults to "reference" for voices and codes and to "embedding" for clips and embeddings. speaker_cfg_scale (default 1.0) above 1 strengthens the voice through classifier-free guidance (a second LM pass per step). Encoding a raw clip into codec codes needs the codec encoder of the native runtimes; here clips clone through the speaker embedding.

import soundfile as sf

wav, sr = sf.read("my_voice.wav", dtype="float32", always_2d=True)
out = model.generate("Cloned from a clip.", speaker_audio=wav.T, speaker_sample_rate=sr, seed=1)

emb = model.embed_speaker(wav.T, sr)  # reuse it across calls
out = model.generate("Same voice, from its embedding.", speaker_embedding=emb, seed=1)

Voice	Language	Style	Gender
Cillian	English	Irish	male
Imogen	English	British	female
Jett	English	energetic	male
Morgana	English	witch	female
Rupert	English	British	male
Sakura	English	anime	female
Savannah	English	American	female
Thaddeus	English	announcer	male
Willow	English	ASMR	female
Wyatt	English	American	male
Gaspard	French	—	male
Liesel	German	—	female
Wolfgang	German	—	male
Ananya	Hindi	—	female
Arjun	Hindi	—	male
Ginevra	Italian	—	female
Lorenzo	Italian	—	male
Minjun	Korean	—	male
Seoyeon	Korean	—	female
Haoran	Mandarin Chinese	energetic	male
Meilin	Mandarin Chinese	energetic	female
Leonor	Portuguese	—	female
Thiago	Portuguese	—	male
Rodrigo	Spanish	—	male
Ximena	Spanish	—	female
Chaitanya	Telugu	—	male
Sravya	Telugu	—	female

Options

Argument	Default	Meaning
language	"en_us"	en_us, en_gb, fr_fr, de, es, it, pt_br, ja, cmn, ko: the text-normalization grammar
text_normalization	True	zyphra_textnorm when installed (otherwise a one-time warning and line breaks collapsed only)
condition_values	unknown	{"loudness": LUFS, "fmax": Hz, "speaking_rate": UTF-8 bytes per second} (or a list of 3)
condition_buckets	unknown	the same as bucket ids (loudness / fmax 0..9, speaking rate 0..29)
temperature, topk, top_p, min_p	0.75, 80, 0.69, 0.0	sampling
repetition_penalty, repetition_window, repetition_codebooks	2.4, 96, 6	repetition penalty
silence_prefill_frames	3	codec silence frames opening the take (0..30)
decode_context, decode_context_frames	False, 30	decode the reference tail as left context (reference mode)
max_new_frames	context left	frame budget (15 frames per second)
seed	random	the take's seed (returned in out.seed)
headroom_db, fade_out_ms	3.0, 0.0	output gain (then a soft limiter at -1 dBFS) and fade-out

generate returns waveform (float32 [samples]), sampling_rate (44100), seed and codes (the decoded codec frames [frames, 10]). model.decode_codes(codes) turns codes into audio; model(input_ids=rows, cache=cache) runs one LM step over prompt rows [rows, 11] for custom decoding loops.

Notes

• Faithful to the reference implementation of the ZONOS3 repository (prompt layout, sampler, EOS handling, codec and speaker encoder op for op). The LM is vectorized (chunked gated-delta rule, grouped experts), so its logits match the reference within float tolerance and long sampled takes may drift from it.
• Not included: speech editing, steering controls and accents, LACI, phoneme input, tag CFG, streaming decode and the codec encoder. The native runtimes serve them.
• The text-to-speech pipeline is not supported: it tokenizes before the model sees the text, which skips the text normalization. Call generate with the raw text.

License

The weights are released under the Apache License 2.0. The remote code is ported from the ZONOS3 repository (MIT) and includes adapted code from AntSpeaker (Apache-2.0), Hugging Face Transformers' Qwen3-Next model (Apache-2.0) and torchaudio (BSD-2-Clause); see NOTICE and licenses/.