embeddinggemma-2-ane-lin8

A Core ML build of google/embeddinggemma-2 for the Apple Neural Engine, for use with Krill (v0.27.1 or later). It has int8 per-channel weights and four fixed-length buckets (64, 128, 256, 512 tokens). Every compute op is placed on the Neural Engine.

Parity with the GPU build served through Krill: cosine min 0.9993 on queries and 0.9989 on passages (against the bf16 build).

Use

krill pull embeddinggemma-2-ane-lin8
krill pull embeddinggemma-2-mlx-4bit-dyn-text
echo 'embed_device = "ane"' >> ~/.krill/config.toml    # or KRILL_EMBED_DEVICE=ane
krill serve
curl -s localhost:57455/v1/embeddings -d '{"model":"embeddinggemma-2-mlx-4bit-dyn-text","input":"hi"}'
# x-krill-embed-device: ane

Serves text up to 512 tokens on the Neural Engine; longer text and image, audio or video inputs fall back to the MLX build. Pull the source model alongside this package (the command above does).

Requires Apple Silicon and macOS. Details: docs/ANE_EMBEDDER.md.