FilBona/embeddinggemma-2-coreml-ane

EmbeddingGemma 2 · Core ML for the Apple Neural Engine

Core ML conversion of the text part of google/embeddinggemma-2, used for on-device semantic search in the iOS app Voce Locale. It runs on the Apple Neural Engine, so it also works while the app is in the background (where the GPU is not available on iOS).

This is a converted and quantized derivative of Google's model, redistributed under the same Apache 2.0 license (see LICENSE). It is not an official Google release.

Files

File	What it is
EmbeddingGemma2.mlmodelc/	Compiled Core ML model (8-bit weights, fixed length 512)
tokenizer.json, tokenizer_config.json	The original tokenizer of google/embeddinggemma-2, unchanged
conversion/	Scripts used for the conversion (reference.py, convert.py, quantize.py, check.py)

Inputs and outputs

• Input input_ids: int32 [1, 512], padded with 0.
• Output embedding: float32 [1, 768], mean-pooled over the tokens and L2-normalized.

The length is fixed at 512 tokens: with flexible shapes Core ML falls back to the CPU (30–40× slower). At 512 tokens the model's sliding window (512) covers the whole input, so the attention mask only excludes padding.

Prompts, as for the original model:

• query: task: search result | query: …
• document: title: <title or none> | text: …

Tokenize with the original tokenizer.json (e.g. swift-transformers or Hugging Face tokenizers); the token ids match the Python reference exactly.

Conversion

Changes made for Core ML, mathematically identical to the original: attention masks built inside the model, attention with batch 1, rotate_half with chunk, repeat_kv with repeat_interleave. Then 8-bit linear weight quantization.

uv venv --python 3.12 .venv
uv pip install --python .venv/bin/python "torch==2.7.0" torchvision==0.22.0 transformers sentence-transformers coremltools numpy pillow librosa
.venv/bin/python conversion/reference.py   # reference vectors (ref.npy)
.venv/bin/python conversion/convert.py     # EG2fixed512.mlpackage (fp16)
.venv/bin/python conversion/quantize.py    # EG2-512-int8.mlpackage
.venv/bin/python conversion/check.py EG2-512-int8.mlpackage 512
xcrun coremlcompiler compile EG2-512-int8.mlpackage out

Measurements (Mac, Apple Silicon, Neural Engine)

Version	Size	Cosine vs original	Time per 512-token chunk
fp16, flexible shape	518 MB	0.999	794 ms (CPU)
fp16, fixed 512	518 MB	0.999	28 ms
8-bit, fixed 512 (this repo)	262 MB	0.998–1.000	27 ms

SHA-256

3a4cf264e8322b06dcbfa94f34ff1e5160bbc03d82353db163d2cf623f570e62  EmbeddingGemma2.mlmodelc/analytics/coremldata.bin
b7fc9952cf7e7dcf9b3dcb0b9c7e62ba593dd966c3cf57b2680ea0f1f2272fd0  EmbeddingGemma2.mlmodelc/coremldata.bin
2d1b06ab8f080f03c01c7d898ebf6426d8ffaa642129a3f1c4f9e89769414be8  EmbeddingGemma2.mlmodelc/metadata.json
2fd0524dd86ae24f6b5c767beec120dc59254e18a98e410d7749a3721d03a9a0  EmbeddingGemma2.mlmodelc/model.mil
905dafd9e926250284774fe8561dfe41222f4963a987b8dad04a1ed01f162b9f  EmbeddingGemma2.mlmodelc/weights/weight.bin
4d777ef5bdc1aa36227abdfb77c3e49e7b9c892d16e1b6bda41c393504828be4  tokenizer.json
17bd5d6e9364ca49a534e1502076593317c298d4a663623091ed45388f004874  tokenizer_config.json

License

Apache License 2.0, as the original model. Original model: © Google, google/embeddinggemma-2.