cortexist/embeddinggemma-2-text-q0f32-MLC

EmbeddingGemma 2 text encoder — MLC q0f32

Text-only extraction of EmbeddingGemma 2: 271,002,648 parameters in FP32. Vision and audio encoders are excluded. The supplied library uses a fixed padded 128-token bucket, including BOS 2 and EOS 1. The fork applies masked mean pooling and L2 normalization, with dimensions 128/256/512/768. Inputs over 128 tokens are rejected rather than silently truncated. Upstream FP32 CPU reference checks pass; do not substitute FP16, which was rejected for this port.

Required runtime

Use cortexist/mlc-llm, our MLC-LLM fork, for model definitions, conversion, builds, and serving. See the fork README and Xavier deployment guide. These custom model exports require the matching fork; stock upstream MLC-LLM is not a qualified runtime for these packages.

libs/ contains prebuilt Linux aarch64, Jetson AGX Xavier (Volta sm_72), JetPack 5 / CUDA 11.4 libraries. They depend on the matching fork's TVM/MLC runtime and are not portable to arbitrary GPUs, operating systems or runtime ABIs. Other devices require recompilation. Source baseline for the published MTP implementation: fd3a5891b8a78f7d50829d0919cbb27e40c26e28. Model-specific integration and reproduction commands are in the fork.

Download with:

hf download cortexist/embeddinggemma-2-text-q0f32-MLC --local-dir models/embeddinggemma-2-text-q0f32-MLC

The root contains converted parameter shards, tensor-cache.json, tokenizer files and mlc-chat-config.json. tensor-cache-b16.json, where present, is retained as the original conversion metadata; tensor-cache.json is the active runtime layout. Extra configurations in configs/ share the root weights; copy the selected configuration into a separate model directory linked to these weights when following the fork's deployment scripts. Model weights alone do not supply an HTTP server.

Related models

This encoder runs independently of the generative model and speculative heads.

Provenance and limitations

Upstream: google/embeddinggemma-2, model-card revision f9972ac22e696a105c482202646979e33e4b5a1e. The upstream card is preserved in UPSTREAM_README.md; the revision identifies the retrieved card, not an independently proven original download revision. manifest.json records exact published file sizes and SHA-256 checksums.

These are conversions of existing checkpoints, not newly trained models. Evaluation applies only to the recorded conditions. Full context capacity is a configuration limit, not a long-context quality or stability guarantee. Consult the fork for benchmark conditions and known limitations. Original model terms and attribution are preserved in the accompanying license and notice files.