josscii/parakeet-0.6b-ja-coreml

Parakeet TDT-CTC 0.6B Japanese — CoreML (A15-ANE-compatible palettized encoder)

Drop-in replacement for FluidInference/parakeet-0.6b-ja-coreml (a CoreML conversion of nvidia/parakeet-tdt_ctc-0.6b-ja, CC-BY-4.0). File names and layout are identical; only the encoder quantization differs.

Why this repack exists

The upstream Encoder.mlmodelc uses INT8 per-channel affine quantization (constexpr_affine_dequantize). On A15-generation Apple Neural Engines it fails to compile/load on the ANE and Core ML silently falls back to CPU, making transcription an order of magnitude slower.

Observed failure matrix on iPad mini 6 (A15, iPadOS 18.7; also reproduced on an A15 device on iOS 26). The same-architecture English encoder (parakeet-tdt-0.6b-v3-coreml, 6-bit LUT, 445 MB) compiles and runs on the ANE of the same devices; op histograms of that encoder and this repo's 6-bit encoder are identical (constexpr_lut_to_dense ×294, iOS17 opset).

Encoder variant	Format	Size	A15 ANE result
upstream int8, iOS17 opset	affine per-channel	594 MB	❌ E5RT: ANE model load has failed ... Must re-compile the E5 bundle. (13)
8-bit LUT, iOS17 opset (Encoder_8bit.mlmodelc)	palettized per-tensor	566 MB	❌ ANECCompile() FAILED (11)
6-bit LUT, iOS17 opset	palettized per-tensor	425 MB	❌ ANECCompile() FAILED (11)
6-bit LUT, iOS16 opset (spec 7)	palettized per-tensor	425 MB	❌ ANECCompile() FAILED (11)
6-bit LUT, v3-shape graph (Encoder.mlmodelc, this release)	palettized per-tensor	425 MB	biases split out of linear/conv + xscale folded — the traced graph now matches the structural class of the v3-en encoder that compiles on A15

What changed

Encoder.mlmodelc was re-created from the original checkpoint (not re-quantized from the released int8 weights):

1. Exported nvidia/parakeet-tdt_ctc-0.6b-ja (.nemo) to a float16 ML Program with the mobius pipeline (models/stt/parakeet-ctc-0.6b-ja/coreml, coremltools 9.0b1, torch 2.7.0, iOS 17 minimum target) — the same pipeline that produced the upstream repo.
2. Graph surgery to match the v3-en structural class (mathematically exact, parity 3.5e-06): the ja checkpoint has real biases on every Linear/Conv and an xscale multiply — the only structural differences vs the bias-less v3-en encoder that compiles fine on the same devices. Biases are split into explicit add ops (bias-fusion passes disabled), xscale is folded into the pre_encode linear, and the output tensor is named encoder (FluidAudio's expected key; earlier artifacts wrongly used encoder_output).
3. Palettized the fp16 weights with palettize_weights(mode="kmeans", nbits=6, granularity="per_tensor") (coremltools 9.0) — the exact recipe of the v3 English encoder.

Encoder output fidelity vs the fp16 reference (random mel input, CPU):

Encoder	SNR	Cosine
upstream int8 (CPU-only on A15)	29.5 dB	0.99944
Encoder_8bit.mlmodelc (fails A15 ANE)	26.0 dB	0.99875
Encoder.mlmodelc 6-bit (this release)	18.7 dB	0.99326

6-bit per-tensor palettization is the same quality tier FluidInference ships as the default encoder of parakeet-tdt-0.6b-v3-coreml.

All other components (Preprocessor, Decoderv2, Jointerv2, CtcDecoder, vocab.json, config.json, metadata.json) are unchanged copies of the upstream repo. Encoder_8bit.mlmodelc is kept for reference and for chips whose ANE compiler accepts it (verified on M-series/macOS 26); FluidAudio ignores it.

Usage

Works as a drop-in with FluidAudio: point the parakeetJa repo at this model — no code changes needed beyond the repo path.