okayuji/embeddinggemma-2-coreai

EmbeddingGemma 2 for Core AI: search text and images locally

Search images and text on iPhone or Mac with the text and image encoders of Google's EmbeddingGemma 2. Text up to 128 tokens and an image become vectors in the same 768-dimensional space.

Start with fp16/: 913.7 MB, checked on iPhone and Mac. int4-int8/ is experimental: 341.9 MB with measured quality loss; it is post-training quantization, not Google's quantization-aware LiteRT-LM build. The full comparison is below.

日本語の要約 · Swift sample app · Image search guide · Camera object finder · iPhone camera demo on X

For: developers building local text/image search or comparing these two precisions on the same inputs. The public Swift runtime supplies the tokenizer, vocabulary lookup and image-patch steps; custom integrations must implement the same contract. Not included: audio, video, text over 128 tokens or more than 70 image tokens. Google's own INT4/INT8 release is LiteRT-LM. Requires: iOS 27 or macOS 27 on Apple silicon; the iOS simulator cannot run Core AI. FP16 was checked on iPhone 17 Pro and M4 Max; INT4/INT8 inference was checked only on Mac.

Quick start

Install the Hugging Face CLI, download the default variant and open the sample app:

git clone https://github.com/oka-yuji/coreai-samples.git
hf download okayuji/embeddinggemma-2-coreai --include "fp16/*" --local-dir ./embeddinggemma-2-coreai
open coreai-samples/CoreAISamples.xcodeproj

On Mac, run CoreAISamples on My Mac, select EmbeddingGemma 2 with the description Find selected images with your own words, choose Open model → Open model folder…, and select embeddinggemma-2-coreai/fp16. Choose Use sample images, wait for indexing, enter Tomatoes and white cheese, then Search. The setup guide covers Xcode, local signing and the separate iPhone Models route. Select the variant folder, not the repository root.

The October 17, 2026 camera recording shows changing text labels while searching camera regions. It is a demonstration, not a controlled retrieval benchmark or evidence that the experimental INT4/INT8 variant passed iPhone acceptance. The object-finder guide records the workflow and remaining device checks.

Prefer a command-line text example? The executed Python example below runs both variants and retains the exact code and recorded environment. The public Swift repository includes text/image preprocessing, the conversion recipe and the measurement records.

Sample output

A text query against five sample photos of refrigerator contents, each named after the dish its ingredients suit, in the companion app's image search (task: search result | query: prefix, cosine similarity, the app hides scores below 0.67). Query: Tomatoes and white cheese.

Rank	iPhone 17 Pro, fp16/	Mac, fp16/	Mac, int4-int8/
1	05-caprese.png 0.7100	05-caprese.png 0.7103	05-caprese.png 0.6795
2	04-gratin.png 0.6655	04-gratin.png 0.6654	04-gratin.png 0.6279
3	01-oyakodon.png 0.6570	01-oyakodon.png 0.6567	01-oyakodon.png 0.6255
4	03-omelette.png 0.6557	03-omelette.png 0.6555	03-omelette.png 0.6038
5	02-ginger-pork.png 0.6433	02-ginger-pork.png 0.6443	02-ginger-pork.png 0.6016

Scores are rounded to 4 decimals. 05-caprese.png is the expected image; all three runs show only it. The search, including tokenization, the text encoder and the ranking of five stored vectors, took 0.0182 s on the iPhone and 0.0146 s on the Mac (fp16/, one run each, not a benchmark). iPhone: iOS 27.0.1 (24A446). Mac: M4 Max 128 GB, macOS 27.0 (26A428). Sources: iPhone from docs/validation/embeddinggemma2-ready-20261016/iphone-searches.json, Mac from docs/validation/embeddinggemma2-int4-int8-20261018/results.json (app_comparison), in coreai-samples.

Key numbers

Conditions: 20 fixed inputs (12 Japanese and English texts, 8 images: five food photos and three gradients), 128 text tokens, 70 image tokens, 768 dimensions, text and images prepared in Swift by the companion app, Core AI with GPU preferred. Mac: M4 Max 128 GB, macOS 27.0 (26A428), Xcode 27.0. iPhone: iPhone 17 Pro, iOS 27.0.1 (24A446). The pass bar for every comparison is cosine at least 0.999, largest difference at most 0.01, finite values, and norm within 0.001 of 1.

	fp16/	int4-int8/	Source
Download size	913,677,333 B (13 files)	341,880,041 B (16 files), 62.58% smaller	int4-int8-20261018.md
Against the original FP32 model, Mac	20/20 pass; lowest cosine 0.999483553; largest difference 0.004896015	0/20 pass; lowest cosine 0.947864461; largest difference 0.047793117	ready-20261016.md; int4-int8-20261018 results.json
Against the original FP32 model, iPhone 17 Pro	20/20 pass; lowest cosine 0.999489425; largest difference 0.004789203	not run (the device was locked)	ready-20261016.md; int4-int8-20261018.md
Against the same quantization run in PyTorch, Mac	(not applicable)	20/20 pass; lowest cosine 0.999257008; largest difference 0.004574524	int4-int8-20261018.md
Same top image as the original model, 12 texts against 8 images	(not reported)	7/12	int4-int8-20261018.md
App search, 3 original examples, Mac	3/3	1/3	int4-int8-20261018.md
App search, 6 held-out queries, Mac	5/6	5/6	int4-int8-20261018.md
App search, iPhone 17 Pro (3 original, 6 held-out)	3/3, 5/6	not run	ready-20261016.md

Source records: FP16 device and app checks, INT4/INT8 comparison, native input verification, and conversion pipeline. Short source names in the table expand to docs/validation/embeddinggemma2-<name> in that repository. The int4-int8/ folder passes against its own PyTorch reference, so the conversion of the quantized weights is faithful; the loss is from the quantization itself. File size is not memory use; memory use is not reported in these records.

Text-only Python example

Run the recorded Python example with both precision variants

On a Mac with macOS 27 and Python 3.11 (coreai-core 1.0.0b3 accepts Python 3.11 to 3.14), save embed.py from below in an empty folder, then run:

python3 -m venv eg2 && source eg2/bin/activate
pip install "coreai-core==1.0.0b3" "tokenizers==0.23.3" "numpy==2.4.6" "huggingface_hub==2.2.0"
hf download okayuji/embeddinggemma-2-coreai --local-dir embeddinggemma-2-coreai
python3 embed.py embeddinggemma-2-coreai/fp16
python3 embed.py embeddinggemma-2-coreai/int4-int8

Each run prints one line per document, the cosine similarity first, highest first. These commands were run as written on an M4 Max Mac with macOS 27.0 (26A428) on 2026-10-19. Both folders ranked the pipe document first, as expected, then A photo of a red toolbox, then Tomatoes and white cheese.

embed.py checks every file against the folder's embedding-manifest.json, loads text.aimodel with the GPU preferred, and ranks three documents for one query. It covers text only; images need the preprocessing in Input contract.

import asyncio, hashlib, json, pathlib, sys
import numpy as np
from tokenizers import Tokenizer
from coreai.runtime import AIModel, ComputeUnitKind, NDArray, SpecializationOptions

root = pathlib.Path(sys.argv[1])
manifest = json.loads((root / "embedding-manifest.json").read_text())
for name, digest in manifest["files"].items():
    assert hashlib.sha256((root / name).read_bytes()).hexdigest() == digest, name
tokenizer = Tokenizer.from_file(str(root / "tokenizer.json"))
scale = np.float32(512 ** 0.5)
if manifest["precision"] == "int4-int8-ptq-s128-v70":
    packed = np.fromfile(root / "token_embeddings.int4", np.uint8).reshape(-1, 256)
    row_scales = np.fromfile(root / "token_embedding_scales.f32", "<f4")
    def rows(ids):
        values = np.stack([packed[ids] & 15, packed[ids] >> 4], -1).reshape(len(ids), 512).astype(np.float32)
        return np.where(values >= 8, values - 16, values) * row_scales[ids, None] * scale
else:
    table = np.fromfile(root / "token_embeddings.bf16", "<u2").reshape(-1, 512)
    def rows(ids):
        return (table[ids].astype(np.uint32) << 16).view(np.float32) * scale

async def main():
    options = SpecializationOptions.from_preferred_compute_unit_kind(ComputeUnitKind.gpu())
    model = await AIModel.load(root / "text.aimodel", specialization_options=options)
    text = model.load_function("main")
    async def embed(prefix, value):
        ids = tokenizer.encode(prefix + value).ids
        if len(ids) > 128:
            raise ValueError("more than 128 tokens")
        mask = np.zeros((1, 128), np.float16)
        mask[0, :len(ids)] = 1
        inputs = rows(ids + [0] * (128 - len(ids))).astype(np.float16)[None]
        output = await text(inputs={"inputs_embeds": NDArray(inputs), "attention_mask": NDArray(mask)}, state={})
        vector = output["embedding"].numpy()[0].astype(np.float32)
        return vector / np.linalg.norm(vector)
    query = await embed("task: search result | query: ", "How do I check the pipe connection safely?")
    documents = ["Turn off the power before checking the pipe connection.",
                 "Tomatoes and white cheese", "A photo of a red toolbox"]
    scores = [float(query @ await embed("title: none | text: ", document)) for document in documents]
    for score, document in sorted(zip(scores, documents), reverse=True):
        print(f"{score:.4f}  {document}")

asyncio.run(main())

How it works

• text.aimodel takes the 128 token vectors of a text and its attention mask, and returns the pooled, L2-normalized 768-number embedding. The vocabulary table stays outside the model: the host looks up each token.
• vision.aimodel turns one image into up to 70 soft tokens of 512 numbers. The host places them in the token sequence <bos> <|image> soft tokens <image|> <eos> and runs text.aimodel, so images and texts share one space.
• The original model card says to run in bfloat16 or float32 and not float16, because float16 can return NaN or silently degraded embeddings. This conversion keeps FP16 by scaling RMSNorm and the final L2 normalization, and gives attention its keys and values as full tensors. The 20-input check in Key numbers is the evidence for this build, not a guarantee for every input. Converted with coreai-torch 0.4.3, coreai-core 1.0.0b3, torch 2.9.0 and Transformers 5.19.0.
• int4-int8/ is a weight-only post-training quantization made with coreai-opt 0.3.0: text projections and the vocabulary are INT4, vision projections and the learned position table are INT8, and norms, biases and scalars keep floating precision. Activations use the same FP16 path. It is not Google's quantization-aware checkpoint.

Input contract

Text:

• Prefix queries with task: search result | query: and documents with title: none | text: , each with one trailing space (the SearchQuery and Document prompts of the upstream config_sentence_transformers.json).
• Tokenize with tokenizer.json; it adds <bos> (2) and <eos> (1). The prefix and both special tokens count toward the 128 tokens. Pad to 128 with token 0 and set the mask to 1 for real tokens and 0 for padding.
• fp16/token_embeddings.bf16: 262,144 rows of 512 little-endian BF16 values. Multiply each value by sqrt(512).
• int4-int8/token_embeddings.int4: 262,144 rows of 256 bytes, two signed 4-bit values per byte, low half first, values -7 to 7. token_embedding_scales.f32 holds one little-endian FP32 scale per row. Value = 4-bit value × row scale × sqrt(512).

Images (the companion app's Swift port of the upstream processor with 70 image tokens):

• Apply the EXIF orientation and use RGB. Transparent and animated images are rejected, and so are images over 40 million pixels.
• Resize with antialiased separable bicubic interpolation to floor(w × f / 48) × 48 by floor(h × f / 48) × 48, where f = sqrt(161280 / (w × h)), so there are at most 630 patches of 16 × 16 pixels. A side that rounds to 0 becomes 48, and the other side becomes (long side ÷ short side, rounded down) × 48, at most 3,360.
• Patch n is the n-th 16 × 16 block in row order. Its 768 values are its pixels in row order, R, G, B each, divided by 255. Its position is (column, row). Unused patches are zeros with position (-1, -1).
• Each image token averages one 3 × 3 block of patches: weight 1/9 at token (row / 3) × (columns / 3) + column / 3.
• Token sequence: [2, 255999] + [258880] × N + [258882, 1] with N = patches / 9; the k-th soft token replaces the vocabulary vector at the k-th 258880.

Function main	Inputs	Output
text.aimodel	inputs_embeds [1, 128, 512] FP16, attention_mask [1, 128] FP16	embedding [1, 768] FP16
vision.aimodel	pixel_values [1, 630, 768] FP16, image_position_ids [1, 630, 2] Int32, pooling_weights [1, 70, 630] FP16	soft_tokens [1, 70, 512] FP16

These values come from the function signatures of the assets and the companion Swift runtime. Compare vectors with cosine similarity. A score is not a probability.

Files

Both folders have the same layout. embedding-manifest.json lists the SHA-256 of every other file and names the precision profile (fp16-safe-s128-v70 or int4-int8-ptq-s128-v70) and the upstream revision.

File	fp16/	int4-int8/
text.aimodel/main.mlirb	277,237,425 B	72,639,389 B
vision.aimodel/main.mlirb	335,796,665 B	168,433,544 B
text.aimodel/ and vision.aimodel/ metadata.json, main.hash	408 + 32 B, 410 + 32 B	408 + 32 B, 410 + 32 B
Vocabulary table	token_embeddings.bf16 268,435,456 B (exact copy of the original BF16 table)	token_embeddings.int4 67,108,864 B + token_embedding_scales.f32 1,048,576 B
tokenizer.json, tokenizer_config.json (upstream, unchanged)	32,170,510 B, 1,599 B	32,170,510 B, 1,599 B
quantized-reference.json (vectors of the 20 inputs from the PyTorch quantization)		341,671 B
quantization-quality.json (differences from the original precision)		99,783 B
README.md (the upstream model card, unchanged)	21,730 B	21,730 B
LICENSE (Apache License 2.0), MODEL_LICENSE.txt (source and changes)	11,358 B, 429 B	11,358 B, 549 B
embedding-manifest.json	1,279 B	1,586 B

The companion app expects one folder as is: fp16/ or int4-int8/, not the repository root.

Operational notes and troubleshooting

• First load is slower. Core AI specializes the model for the device on first load. In an earlier record (Mac, 2026-10-15, an earlier package of the same FP16 profile), the first indexing of five images took 11.637 s and a warm run 0.767 s, preprocessing included (embeddinggemma2-native-20261015.md).
• Keep the folder unchanged. The companion app checks every file before loading and stops with Checksum mismatch: <file> for a changed file, Unlisted asset file: <file> for an extra file inside text.aimodel/ or vision.aimodel/ (such as a .DS_Store written by Finder), and Symbolic links are not supported in bundles for a link.
• Long text. More than 128 tokens, prefix included, is rejected; it is not truncated.
• Quick start. embed.py stops with AssertionError: <file> when a file does not match the manifest, and with FileNotFoundError: [Errno 2] No such file or directory: '<path>' when a listed file is missing. Download that folder again.

Limitations

• int4-int8/ is less accurate. Against the original model none of the 20 inputs meets the bar above (lowest cosine 0.947864461), 7 of 12 texts find the same top image, and 1 of the 3 original app examples passes. A Japanese food query (トマトと白いチーズが並んでいる写真, "a photo of tomatoes and white cheese side by side") still ranks the right photo first but scores 0.6688, below the app's 0.67, so it is hidden. An egg query's top result changed from the omelette to the oyakodon.
• int4-int8/ was installed on the iPhone 17 Pro, but inference there was not run because the device was locked. Its quality, speed, heat and memory on iPhone are unknown.
• fp16/ on iPhone was checked by the 20 inputs and the app checks above. Long use, memory, and touch use of the app were not measured.
• Memory use is not reported in these records for either folder, and speed has no controlled benchmark. FP16 and INT4/INT8 speed were not compared.
• The search results come from five similar photos and nine queries. They are not a general retrieval benchmark. The 0.67 display threshold was set on six calibration queries; on six other queries it gave 5/6, and one relevant query scored 0.660 and was hidden.
• Only text, images at 70 tokens, and 768 dimensions were converted and checked. Audio, video, the upstream default of 280 image tokens, texts over 128 tokens, and shorter Matryoshka dimensions were not.
• Every measurement used GPU-preferred specialization. Placement on the Neural Engine was not measured.

Verification

• Conversion. The export step checks its wrapper against the original FP32 reference before writing each asset (embeddinggemma2-pipeline-20261016.md). The final check runs the converted files from Swift on raw texts and images and compares them with the original FP32 model under the bar in Key numbers. The 20 vectors were recalculated independently with math.fsum (embeddinggemma2-native-20261015.md).
• Tokens and pixels. The Swift tokenizer gave the same token IDs as the Python reference for all 12 texts, and the same token counts for all 8 images. Pixels differed by at most 1/255 for the five photos and 0 for the gradients (embeddinggemma2-native-20261015.md).
• Devices. The same fp16/ files (manifest SHA-256 1a1122d9…) were checked on the Mac and the iPhone 17 Pro. For int4-int8/, the manifest copied back from the iPhone matched (7c37ea82…).
• Quantization. int4-int8/ was checked against a PyTorch run of the same quantization and, separately, against the original model; both results are in Key numbers and in quantization-quality.json.
• This repository. Every file of both folders was checked against the built files by size and SHA-256 and against each folder's manifest.

License and attribution

The weights are a conversion of google/embeddinggemma-2 (revision 914f7f89142e33e77833254d9c9b90c3cef7303b) by Google DeepMind, released under the Apache License 2.0; the same license applies to this conversion, and its text is in fp16/LICENSE and int4-int8/LICENSE. Changes from the original: the text and image encoders were converted to Core AI with the FP16 changes above, for 128 text tokens and 70 image tokens; the audio encoder is not included; the vocabulary table is stored outside the model (an exact BF16 copy in fp16/, INT4 values with FP32 row scales in int4-int8/); and the int4-int8/ weights are quantized. Each folder's MODEL_LICENSE.txt records the source and changes. The tokenizer files and the original model card (README.md in each folder) are unchanged. The original model card states: "Deployments must adhere to the Gemma Prohibited Use Policy."

Product names are used only to identify the source model and the platforms. Converted and verified by Yuji Oka (okayuji). This conversion is an independent work and is not affiliated with or endorsed by Google or Apple.

日本語の要約

• google/embeddinggemma-2 の文と画像のエンコーダを Core AI (iOS 27 / macOS 27)に変換したものです。文(128 トークンまで)と画像を同じ 768 次元のベクトルにします。 音声・動画は含みません。
• fp16/(913,677,333 B)が既定版です。元の FP32 との比較で固定の 20 入力がすべて基準を満たしました (Mac 最小 cosine 0.999483553、iPhone 17 Pro 0.999489425)。
• int4-int8/(341,880,041 B、62.58% 小さい)は実験的な事後量子化です。同じ量子化の PyTorch とは 20/20 で 一致しますが、元の精度とは最小 cosine 0.947864461 で、アプリの元の 3 例は 1/3(FP16 は 3/3)でした。 iPhone での推論は端末がロックされていたため実行していません。
• メモリ使用量はこれらの記録にありません。FP16 と INT4/INT8 の速度は比較していません。
• ライセンスは Apache License 2.0 です。元のモデルカードは Gemma Prohibited Use Policy の順守を求めています。