samthehugenerd/daphne-kokoro-coreml

Daphne — Kokoro CoreML (Apple Neural Engine)

Daphne is a house-assistant voice: a German Kokoro fine-tune read through British English G2P. This repo is her as a drop-in 7-stage CoreML chain for FluidAudio's KokoroAne backend, so she runs on the ANE on iPhone, iPad, Apple Watch and Mac instead of over the network.

Synthesis runs at roughly 8–20× realtime on an M4 Pro.

The recipe

The model alone is not the voice. Three things have to be applied together, and dropping any one of them gives a recognisably different speaker:

1. British G2P. The checkpoint is German; the accent on English text comes from reading it through a British lexicon. FluidAudio's bundled English frontend is American, so install misaki's gb_gold + gb_silver as a custom lexicon — it is consulted before the bundled one, leaving only out-of-vocabulary words on the American fallback.
2. speed = 1 / 1.1 at synthesis.
3. Play the result at 26400 Hz. Kokoro has no pitch control, so pitch is applied by resampling and asking the model for speed / pitch to put the duration back where it started, netting +10% pitch at the original tempo. Reinterpreting the 24 kHz buffer at 26400 Hz is bit-identical to resample_poly(y, 10, 11) and needs no filter, which also means it carries no state across chunk boundaries.

let manager = KokoroAneManager(defaultVoice: "bf_alice", directory: repo)
manager.setEnglishCustomLexicon(gbLexicon)   // word → Misaki IPA
try await manager.initialize()
let wav = try await manager.synthesize(text: line, voice: "bf_alice", speed: 1.0 / 1.1)
// play the 24 kHz buffer at 26400 Hz

Streaming

The chain is not autoregressive — durations are predicted for the whole phoneme string and every frame is generated at once — so nothing arrives early within an utterance. Time-to-first-audio comes from splitting at clause boundaries and starting playback on the first chunk while the rest synthesize behind it.

On an M4 Pro a five-clause reply synthesizes each chunk in 0.08–0.15 s for 1.5–3 s of speech (~20× realtime): TTFA 0.12 s against a per-chunk budget of 2.4 s. Only the first chunk is on the critical path.

She ships in a stock slot

bf_alice.bin is Daphne's style vector, not Alice. FluidAudio validates voice names against a compile-time roster before it looks for a local pack, so a custom name is rejected; riding an unused stock slot is the way in until that check moves.

The pack is the usual flat [510, 256] fp32 layout — [0..<128] is style_timbre, [128..<256] is style_s. Every row is identical, so the phoneme-length bucket does not matter.

What is in here

Stage	Source
KokoroPostAlbert.mlmodelc, KokoroVocoder.mlmodelc	converted from this fine-tune
KokoroAlbert, KokoroAlignment, KokoroProsody, KokoroNoise_v2, KokoroTail_v2	FluidInference/kokoro-82m-coreml, unmodified

Only two stages needed converting. The fine-tune moved decoder and text_encoder and left bert, bert_encoder and predictor bit-identical to stock, so the stages cut from those modules are reused as published — which also keeps FluidAudio's _v2 phase and iSTFT fixes, since Noise and Tail are speaker-independent.

Provenance and licence

Base model hexgrad/Kokoro-82M, Apache-2.0. The German fine-tune underneath (51 speakers, HUI corpus) was published as junkstage/kokoro-deutsch-hui-base, which is no longer on the Hub; its licence terms could not be confirmed at upload time. The CoreML conversion follows laishere/kokoro-coreml.