lineofflight/pocket-tts-coreml

Pocket TTS for Core ML

Kyutai's Pocket TTS, converted to Core ML for Apple Silicon Macs: the 6-layer Dutch, the 6-layer French and English 2026-10. They are the packs voice, a speech tool for coding agents, speaks through on a Mac.

The packs have the layout, inputs and outputs of FluidInference's v2.1 packs in FluidInference/pocket-tts-coreml, which has no Dutch, no 6-layer French and no English 2026-10. They were converted with FluidInference's own recipe.

What's here

constants_bin/mimi_init_state/  Mimi's initial state, a float32 file per tensor, which every pack shares
v2.1/<pack>/
  cond_prefill_ane.mlmodelc     the voice's and text's keys and values into the cache, in one call
  flowlm_step_ane.mlmodelc      the transformer, one frame a call
  flow_decoder_fused.mlmodelc   the flow's 8 Euler steps in one call
  mimi_decoder.mlmodelc         a latent to 80 ms of 24 kHz audio
  constants_bin/
    tokenizer.model             Kyutai's SentencePiece model for the language
    text_embed_table.bin        the text embeddings, float32 [4001, 1024]
    bos_emb.bin                 the BOS latent, float32 [32]
    bos_before_voice.bin        the latent before a voice's prompt, float32 [1024]
    <voice>.safetensors         a voice's KV-cache snapshot, Kyutai's own file

Pack	Kyutai model	Voices	Size
dutch	languages/dutch, 6 layers	daan and the 19 below	487 MB
french	languages/french, 6 layers	daan and the 19 below	487 MB
english_2026-10	languages/english_2026-10, 6 layers	the 19 below	484 MB

english_2026-10 is Kyutai's english_2026-09_2 training recipe retrained with a loader fix for an unaligned last word, so it reads sentence-final numbers. At the revision below it is also Kyutai's languages/english: the two folders hold the same files.

Mimi's initial state is the recipe's own, the same file for file for all three packs and as constants_bin/mimi_init_state/ at the root of FluidInference's repository. It is here too, so the packs need nothing from elsewhere.

FILES.txt lists every file with its sha256 and size.

Models and inputs

Every model takes and gives float32. The first three compute in float16, Mimi in float32.

Model	Inputs	Outputs
cond_prefill_ane	conditioning [1, 256, 1024], valid_len [1], per layer i of 6: k_cache{i}, v_cache{i} [1, 512, 16, 64], position{i} [1]	new_k_cache{i}, new_v_cache{i}, new_position{i}
flowlm_step_ane	sequence [1, 1, 32] (the BOS latent on the first step, never NaN), the caches and positions	transformer_out [1, 1, 1024], is_eos [1, 1, 1] (a logit; stop above -4), the new caches and positions
flow_decoder_fused	transformer_out [1, 1024], latent_init [1, 32]	latent_final [1, 32]
mimi_decoder	latent [1, 32] and 24 state tensors	audio [1, 1, 1920] and the 24 states, under the traced var_ names of FluidInference's packs

The names and shapes match FluidInference's v2.1/english exactly.

Where it comes from

The weights are Kyutai's, from kyutai/pocket-tts-without-voice-cloning at revision a7056f505a4a8ca72d8b5fb1440071d6f1a35739, fetched with curl and checked against the sha256 the repository lists:

File	sha256
languages/dutch/model.safetensors	59b5c3249ab05e8b5fcb0a296e9b0cb0c0a20da5e1fd28c04bb90b2175612aeb
languages/french/model.safetensors	56366cfed6c25bfcd3bf4c61e66e06efaf52e78a5d06e1dcade1c588495bc699
languages/english_2026-10/model.safetensors	4c56cbd171c63e8a26ea61f5d48b8a0cd8f3c51f4652c0ab9a19bfcced8c931d

The tokenizers and the voices are Kyutai's files from the same revision, copied unchanged.

The conversion is FluidInference's mobius recipe, models/tts/pocket_tts/coreml (Apache-2.0), on its branch feat/pocket-tts-flowlm-ane at commit 210d6b6d21177115babfece0de5e1795cffec2a2. That branch carries the _ane and fused converters (its Trials 16 to 21) beside everything on main at the time; main, at 26f5fa8059d5ab3d34449adce6ff6ed86734e522, still makes only the older cond_step, flowlm_step and flow_decoder.

The recipe ran with torch 2.9.1, coremltools 9.0 and Kyutai's pocket-tts 4.0.1, the first two as its uv.lock pins them, and was compiled with xcrun coremlc from Xcode 27.1. As a check, the same steps on Kyutai's english_2026-04 rebuilt FluidInference's own v2.1/english: cond_prefill_ane, flowlm_step_ane and mimi_decoder with identical weights, and all four giving the same outputs, Mimi's within 2e-4 for the reason below.

These edits to the recipe were needed, and are the only ones:

• _language_arg.py: dutch, french and english_2026-10 added to the languages it accepts.
• traceable_mimi_decoder.py: 3 changes for pocket-tts 4, which the newer packs need. Its attention state's new pad (0 at batch 1) is left out, the quantized latent skips the quantizer decode_from_latent now applies itself, and the attention takes the attn_mask argument the layers now pass.
• convert_mimi_decoder.py: its renaming of Mimi's outputs is skipped, so they keep the traced names FluidInference's published packs have.
• mlmodelc_all.sh: the three _ane and fused models added to what it compiles.

Two steps are not in the recipe. bos_before_voice.bin is flow_lm.bos_before_voice written as float32, and Mimi's initial state is the recipe's own TraceableMimiDecoder state, with the NaN cache fill as 0. Both match FluidInference's English pack byte for byte.

Kyutai's pocket-tts has used the tanh approximation of GELU since 3.1.0, and Dutch, the 6-layer French and English 2026-10 came after it. The recipe's transformer models keep exact GELU, as FluidInference's packs do, while Mimi follows the package. The difference is about 1e-3 in the transformer's output, against 2e-2 to 3e-2 from float16, and 2e-4 in Mimi's audio.

The commands

From a checkout of mobius and a scratch directory:

git clone --depth 1 https://github.com/FluidInference/mobius.git mobius
git -C mobius fetch --depth 1 origin "refs/heads/feat/pocket-tts-flowlm-ane:refs/remotes/origin/feat/pocket-tts-flowlm-ane"
git -C mobius worktree add --detach ../mobius-ane origin/feat/pocket-tts-flowlm-ane
uv venv env --python python3.12
uv pip install --python env/bin/python torch==2.9.1 coremltools==9.0 pocket-tts==4.0.1 \
  numpy==2.4.1 safetensors==0.7.0 sentencepiece==0.2.1 scipy==1.17.0 einops==0.8.1 \
  huggingface-hub==1.3.1 pydantic==2.12.5 beartype==0.22.9 pyyaml

Then, for each of dutch, french and english_2026-10, in mobius-ane/models/tts/pocket_tts/coreml, with the model loaded from a local copy of pocket-tts's config for the language whose weights_path is the fetched file:

python convert_models/convert/convert_cond_prefill_ane.py --language L --t-max 256
python convert_models/convert/convert_flowlm_step_ane.py --language L
python convert_models/convert/convert_flow_decoder_fused.py --language L --num-steps 8
MIMI_SEMANTIC_NAMES=0 python convert_models/convert/convert_mimi_decoder.py --language L --compute-precision fp32
python convert_assets/export_constants.py --language L
python convert_assets/pack_constants_bin.py --language L
DEVELOPER_DIR=/Applications/Xcode.app/Contents/Developer LANGUAGES=L ./mlmodelc_all.sh

Each pack is then build/L/'s four .mlmodelc (with metadata.json for Mimi only, as FluidInference's packs have it) and its constants_bin/ files for the voices listed.

Checked

The recipe's own check speaks its 33 to 37 word sample text through each pack and has Whisper large-v3 hear it. Through these models, at the recipe's temperature of 0.7 and at voice's 0.3:

Pack	Voices	Words wrong, of 33 to 37, over 4 runs
dutch	alba, daan	1
french	alba, estelle	0
english_2026-10	alba, charles	0
FluidInference's v2.1/english	alba, charles	0

The recipe has no Dutch sample text, so the Dutch one says what the others do.

Voices

Voice	Source	License
alba	Alba MacKenna	CC BY 4.0
anna, vera, fantine, charles, paul, eponine, azelma, george, mary, jane, michael, eve	VCTK p228, p229, p244, p254, p259, p262, p303, p315, p333, p339, p360, p361	CC BY 4.0
marius, javert	Kyutai voice donations	CC0
estelle	Unmute	CC0
lola, giovanni	Common Voice	CC0
juergen	Kyutai, source not stated	CC BY 4.0 (Kyutai's)
daan	CML-TTS, Dutch	CC BY 4.0

Each voice is Kyutai's snapshot for that language, from languages/<pack>/embeddings/. Kyutai's repository carries the CML-TTS sample cml_tts_nl_11290_9412_000030_0001.flac at its root.

License

Pocket TTS is Kyutai's, under CC BY 4.0, and so are these conversions of it: see kyutai/pocket-tts. Kyutai's model card sets these terms:

Prohibited use: Use of our model must comply with all applicable laws and regulations and must not result in, involve, or facilitate any illegal, harmful, deceptive, fraudulent, or unauthorized activity. Prohibited uses include, without limitation, voice impersonation or cloning without explicit and lawful consent; misinformation, disinformation, or deception (including fake news, fraudulent calls, or presenting generated content as genuine recordings of real people or events); and the generation of unlawful, harmful, libelous, abusive, harassing, discriminatory, hateful, or privacy-invasive content. We disclaim all liability for any non-compliant use.

The conversion recipe is FluidInference's mobius, Apache-2.0. The voices are under the licenses above.