fredrunit/embeddinggemma-2-text-8bit

EmbeddingGemma 2, text only, 8-bit MLX

google/embeddinggemma-2 (revision 914f7f89142e33e77833254d9c9b90c3cef7303b) with its image and audio encoders removed, for text search on Apple silicon. 320 MB in all, against about 1.27 GB for the 8-bit conversion that keeps the encoders.

• Text tower only (271M of the model's 744M parameters).
• Matrices quantized affine, 8 bits, group 64; scales, biases, norms and scalars in bfloat16 (never float16: the model overflows in float16).
• 768-dimensional output with Matryoshka prefixes 512, 256 and 128; mean pooling, L2 normalization. Use the source model's prompts, for example task: search result | query: … and title: none | text: ….
• The tokenizer and the sentence-transformers configs are Google's files, byte for byte.

Converted by RuniT-ChatFramework's Scripts/convert-embeddinggemma2-text.py. Its vectors match the FP32 reference cosines of Google's checkpoint within 0.002 on multilingual query and document pairs, the same as the 8-bit build with encoders.

License: Apache 2.0, as the source model (https://ai.google.dev/gemma/docs/gemma_4_license).