jicmics123/embeddinggemma-2-mlx

EmbeddingGemma 2 (text and vision), quantized for MLX

This is a quantized copy of Google's EmbeddingGemma 2, made for on-device search in the Einso app (https://einso.app).

The model is Google's work and is released under the Apache License 2.0 (included here as LICENSE). This copy is shared under the same licence.

What was changed

• The audio tower (audio_tower.*, embed_audio.*) was left out. The text and vision towers are unchanged in structure.
• Weights were quantized with MLX's affine quantization (group size 64, scales and biases stored beside each weight): the text and vision layers' linear weights at 6 bits, the word embedding table at 4 bits. Each layer's bits can be read from its weight and scales shapes. Norms and other small tensors keep Google's original bfloat16 values.
• config.json is Google's, with a quantization entry added. The tokenizer files are Google's, unchanged.

Quality

We compared this copy with Google's bfloat16 weights on 40 board-style notes, 24 search questions (including one in Spanish) and 5 photos:

Vectors alike (cosine), notes	0.998 on average, 0.996 at worst
Vectors alike (cosine), questions	0.999 on average, 0.998 at worst
Vectors alike (cosine), photos	0.992 at worst
Same best result	24 of 24 questions
Top-3 results shared	97%
Weights file	341 MB (from 878 MB without audio, 1.49 GB with)
Peak memory on a Mac, one batch	about 0.6 GB (from 1.2 GB)

Use

Queries use the prefix task: search result | query: and documents use title: none | text: , as in Google's model card. Output vectors are mean-pooled and L2-normalised, 768 dimensions.