seanll95/chatterbox-nano-coreml

Chatterbox Nano — Core ML

Core ML conversions of ResembleAI/chatterbox-nano, built for on-device text-to-speech on iOS. Used by Annologue, an iOS audiobook player, to synthesise chapters locally.

These run faster than real time on an iPhone, with the two heaviest stages executing almost entirely on the Neural Engine.

Contents

File	Precision	Size	Compute units	Measured device placement
t3_prefill_ane.mlpackage	fp16	173 MB	CPU + GPU	ANE cannot compile this graph (ANECCompile() FAILED); it is ~1 % of wall time
t3_decode_ane.mlpackage	fp16	173 MB	CPU + ANE	525/537 ops = 97.8 % ANE
s3gen_flow.mlpackage	fp16	230 MB	CPU + ANE	3300/3317 ops = 99.5 % ANE
s3gen_vocoder.mlpackage	fp16 (selective)	41 MB	CPU + GPU	179 convolutions over 288 000 samples exceed the ANE's per-dimension limits
t3_embeddings.bin	fp16	86 MB	—	Text/speech/positional embedding tables, gathered on the CPU
nano_conds.bin	—	1.3 MB	—	Default speaker conditionals, with cond_emb precomputed
tokenizer.json, tokenizer_config.json	—	3.4 MB	—	—

.mlpackage files are uncompiled. Compile them on-device with MLModel.compileModel(at:) and cache the result — compilation takes roughly a minute in total and only needs to happen once per install.

How this differs from upstream

T3 is split into two packages. Upstream is one autoregressive transformer; here prefill and decode are separate MLModels with independent MLStates, and the KV prefix is copied across per utterance. This costs +173 MB of disk (the two halves do not share weights) and buys decode its own compute-unit placement, which is the entire reason decode can reach the Neural Engine while prefill cannot.

Decode's attention is written out rather than fused. A fused scaled_dot_product_attention places on the ANE but the ANE silently ignores its mask operand, so attention runs over the whole KV cache including the ~500 slots holding nothing. The output stays fluent and is simply the wrong words. Spelling attention as matmul → scale → add → softmax → matmul makes the mask an ordinary add the compiler cannot drop. This cost no latency and lost no ANE placement.

S3Gen is split into flow and vocoder halves so the flow half can be fp16 on the ANE while the vocoder stays on the GPU. The vocoder's HiFi-GAN upsamplers are re-expressed as sub-pixel (reshape + convolution) rather than transposed convolutions, which is numerically exact and removes ops that fell back to the CPU. Its cumsum phase accumulator is kept in fp32 — that one is a genuine fp16 hazard.

Quantisation stops at fp16. int8 on S3Gen was measured and rejected: log-spectral distance against fp32 went from 0.034 to 0.354, and MLComputePlan stopped loading the model. int8 on T3 halves the weights but was measured at 0.95×/0.99× — a RAM lever, not a speed one.

⚠️ No watermarking

Upstream Chatterbox applies Resemble AI's PerTh watermarker to every generated file. These conversions are the T3 and S3Gen models only — audio generated from them is not watermarked. If you need the watermark, apply perth to the output yourself. Please use these responsibly.

Licence and attribution

MIT, inherited from the upstream model. Copyright © 2025 Resemble AI — see LICENSE. The underlying model, training and weights are Resemble AI's work; this repository contributes only the Core ML conversion.

@misc{chatterboxtts2025,
  author       = {{Resemble AI}},
  title        = {{Chatterbox-TTS}},
  year         = {2025},
  howpublished = {\url{https://github.com/resemble-ai/chatterbox}},
  note         = {GitHub repository}
}