lee-futo/Kokoro-82M-executorch

Kokoro-82M for ExecuTorch (XNNPACK)

An ExecuTorch 1.5.1 export of hexgrad/Kokoro-82M (revision f3ff357) for XNNPACK, the CPU backend. For everything about the model itself, see the original model card. This card only covers the export.

file	weights	size
xnnpack_fp32/kokoro.pte	fp32	325 MB
voices/*.bin	the 28 English voices (af_, am_, bf_, bm_)	15 MB

Running it

One program with two methods. The phoneme axis and the frame axis are both dynamic, and nothing is padded: Kokoro's bidirectional LSTMs and its instance norms over time both change their output under padding.

predict(input_ids [1,N] int64, ref_s [1,256], speed [1])
    -> d [1,N,640], t_en [1,512,N], duration [N] int64
vocode(d, t_en, aln [1,N,F], ref_s, rand_ini [1,9], sine_noise [1,9,600F])
    -> waveform [600F], float32 at 24 kHz

1. Text to phonemes, outside the program. Use misaki, or espeak-ng, then map the phonemes through config.json's vocabulary, dropping any character it does not have, and wrap the ids in 0: [0, *ids, 0]. Up to 512 ids.

2. Pick the style row. voices/<name>.bin is little-endian float32 [510, 256]. The row is len(phonemes) - 1, counted on the phoneme string, as upstream does.

3. Run predict.

4. Build the alignment. aln[n, f] = 1 where frame f falls within phoneme n's duration, starting at the cumulative sum of the earlier durations. There are exactly F = sum(duration) frames, at most 2400. This is why there are two methods: the vocoder's input size is the predictor's output.

5. Run vocode with noise you supply.

   • rand_ini is uniform in [0, 1).
   • sine_noise is standard normal.

   Upstream draws both inside the model. To reproduce it exactly, draw torch.rand(1, 9), then torch.empty_strided((1, 600F, 9), (5400F, 1, 600F)).normal_().transpose(1, 2).

get_sample_rate, get_samples_per_frame, get_max_phonemes, get_max_frames and get_style_dim are constant methods.

What the export changed

All of these are numerically equivalent to upstream:

• LSTMs: the six LSTMs are rolled scan loops, both directions in one loop, so their length stays dynamic. nn.LSTM exports with its length pinned.
• XNNPACK rewrites: weight norm is baked into the weights. ALBERT's GELU, the instance norms, the AdaIN layers and Snake are written so XNNPACK can take them.
• Vocoder noise: taken as inputs instead of drawn inside the model.
• Partitioning: vocode is partitioned without XNNPACK's PermuteConfig. A permute that feeds two consumers produces a partition XNNPACK rejects.

metadata.json records the export, including what XNNPACK delegated. ops.txt lists the portable kernels a selective ExecuTorch build needs. export/ holds the scripts.

Verified

The program was compared with eager PyTorch on 10 held-out English sentences, in af_heart and bm_george, using the same noise:

• Durations: equal on 20 of 20 utterances.
• Log-mel distance: 0.059 dB. Eager against itself with different noise measures 0.469 dB.
• Speed: real-time factor 0.27 on 4 threads and 0.17 on 24, on an i9-12900K.

License

Apache-2.0, as Kokoro-82M (see LICENSE). The program and voice files are converted from hexgrad's Kokoro-82M weights and voice packs.