avenkin-search-eg2-text

The text encoder of EmbeddingGemma 2, converted to Core ML with int8 weights, for on-device search in the Avenkin iOS app. See NOTICE for exactly what changed from the original.

Files

File	What it is
eg2-text-int8.mlpackage	Core ML ML Program. Inputs input_ids (int32, 1 × L) and attention_mask (float32, 1 × L, 1 = token), with L from 1 to 512. Output embedding (float32, 1 × 768): mean-pooled over the tokens, L2-normalised.
tokenizer.json, tokenizer_config.json, config.json	Unmodified from the base model. Tokenize with special tokens (<bos> … <eos>).
LICENSE, NOTICE	Apache 2.0, and the changes made.

Use

• Prefix a question with task: search result | query: and a passage with title: none | text: .
• Shorter vectors: keep the leading 512, 256 or 128 values and re-normalise.
• Run it on the CPU, in float32. In float16, which the Neural Engine uses, every vector moves: the median cosine against the float32 reference was 0.978 on our checks. The base model card warns against float16 too.

Checks (2026-11-07)

• 329 invented sentences (questions, passages, eight in other languages and scripts).
• Against the float32 reference encoder: minimum cosine 0.99977, median 0.99987, with no NaN.
• On an iPhone Air (iOS 27.2, CPU only): a short question in about 40 ms warm, and about 167 MB of memory with the model loaded.
• The tokenizer (swift-transformers) matches the reference on all 329 sentences.