nomic-embed-text-ane-fp16

A Core ML build of nomic-ai/nomic-embed-text-v1.5 for the Apple Neural Engine, for use with Krill (v0.27.1 or later). It has fp16 weights and four fixed-length buckets (64, 128, 256, 512 tokens). Every compute op is placed on the Neural Engine.

Parity with the GPU build served through Krill: cosine min 0.99962 on queries and 0.99939 on passages.

Use

krill pull nomic-embed-text-ane-fp16
krill pull nomic-embed-text
echo 'embed_device = "ane"' >> ~/.krill/config.toml    # or KRILL_EMBED_DEVICE=ane
krill serve
curl -si localhost:57455/v1/embeddings -d '{"model":"nomic-embed-text","input":"hi"}' | grep -i x-krill
# x-krill-embed-device: ane

Serves text up to 512 tokens on the Neural Engine; longer text falls back to the MLX build. Pull the source model alongside this package (the command above does).

Requires Apple Silicon and macOS. Details: docs/ANE_EMBEDDER.md.