Chatterbox Turbo — all-CoreML (ANE-optimized)
On-device Apple Silicon build of ResembleAI Chatterbox Turbo for iOS 18+ / macOS 15+, arm64. The entire pipeline is CoreML — T3 prefill
- decode run fully on the Apple Neural Engine as one weight-shared multifunction model, and the S3Gen audio synth is three CoreML packages (encoder on ANE, CFM + vocoder on the GPU).
Sibling repos: iliasaz/chatterbox-nano-coreml (smaller, faster) and iliasaz/chatterbox-multilingual-coreml (23 languages).
Links
- Upstream model — ResembleAI/chatterbox-turbo by Resemble AI.
- Runtime — iliasaz/chatterbox-coreml, the Swift package + demo app that loads this repo.
- Watermarking — iliasaz/perth-coreml (Swift) / weights. Every utterance the runtime generates is watermarked with Perth, as upstream chatterbox does.
Artifacts
| File | What | Compute |
|---|---|---|
T3LM.mlpackage |
T3 prefill + decode, multifunction (q=W prefill, q=1 decode, shared MLState KV cache), 8-bit palettized |
CoreML — ANE on iPhone, GPU on Mac |
S3Encoder.mlpackage |
S3Gen flow encoder: speech tokens → mel mu |
CoreML — ANE |
S3CFM.mlpackage |
conditional flow-matching Euler solver (turbo: 2-step meanflow) | CoreML — GPU |
S3Vocoder.mlpackage |
mel → 24 kHz waveform (HiFTGen + ISTFT) | CoreML — GPU |
CAMPPlus, MatchaMel, VEMel, VELSTM, S3Tokenizer .mlpackage |
on-device voice-cloning conditioning encoders | CoreML |
text_emb.npy, speech_emb.npy |
embedding tables (host-side lookups) | — |
spkr_enc_weight.npy, spkr_enc_bias.npy |
speaker conditioning linear (256→1024) | — |
default-conds.safetensors, tokenizer files |
default voice + GPT-2 BPE tokenizer | — |
Pipeline
text → GPT-2 BPE → host-assemble inputs_embeds + position_ids + masks
→ T3LM "prefill" function (CoreML, ANE) — writes shared KV state
→ T3LM "decode" function (CoreML, ANE) — autoregressive, reads/writes same state
→ S3Encoder (ANE) → S3CFM (GPU) → S3Vocoder (GPU) — 24 kHz waveform
All attention masks (attn_mask, write_mask, decode update_mask) are built
on the Swift host and passed in — constructing them inside the stateful 24-layer
prefill graph fails the ANE compile, so keeping mask construction off the model is
what makes the prefill ANE-compilable. The prefill's attention is also
hand-decomposed rather than fused: on iPhone the ANE's fused attention kernel
silently drops attn_mask for q_len >> 1.
On-device numbers (iPhone 17 Pro Max, iOS 26.5, warm)
| stage | latency | engine |
|---|---|---|
| T3LM cold load (first launch after install) | ~30 s | ANE AOT compile |
| T3LM warm load (subsequent launches) | ~0.5–1 s | persisted compiled model (cache hit) |
| prefill predict (q=512 one-shot) | ~0.07–0.13 s | ANE |
| decode predict | ~29–35 ms / token | ANE (8-bit) |
| synth (S3Encoder + S3CFM + S3Vocoder) | ~0.8 s | ANE + GPU |
The compiled .mlmodelc is persisted once and reused across launches, so the
one-time ANE compile isn't paid every launch. The runtime also overlaps chunk
N+1 decode (ANE) with chunk N synth (GPU) for faster multi-chunk throughput.
License
MIT. These weights are a CoreML format conversion of ResembleAI/chatterbox-turbo, which Resemble AI publishes under the MIT licence; the conversion inherits those terms, and copyright in the weights stays with Resemble AI — no claim of ownership is made over them here. Redistribute with attribution to Resemble AI.
The Swift runtime that loads them is separately MIT-licensed; see the LICENSE and
NOTICE files in iliasaz/chatterbox-coreml.
- Downloads last month
- 50
Model tree for iliasaz/chatterbox-turbo-coreml
Base model
ResembleAI/chatterbox-turbo