lainsoykaf/WeMM-Embedding-2B-GGUF

WeMM-Embedding-2B GGUF (unofficial)

Unofficial GGUF conversion of tencent/WeMM-Embedding-2B for llama.cpp. It is the default embedding model of imgsearch, a local-first image search app. Not affiliated with or endorsed by Tencent. All credit for the model goes to the WeChat Vision Team; it is licensed under Apache 2.0.

File	Size	SHA-256
WeMM-Embedding-2B-Q4_K_M.gguf	1.56 GB	3c6aa2dab16f250f730e7e87305fbd637a8fd068770d3cae16a0c16e33678d6d
mmproj-WeMM-Embedding-2B-f16.gguf	0.67 GB	02115c4aebe057aaf0473c8c9ee4efbbd080ae1b07386910cb326c1c042baaaa

Usage

Both files are needed for image input. Output: 2048 dimensions, L2-normalize; Matryoshka truncation (re-normalize) to 1024/512/256 works well.

The prompt must match the reference processor output exactly. No system turn, no instruction; queries and documents are encoded the same way. Take the last-token hidden state (pooling none, last position, or --pooling last):

text:  <|im_start|>user\n{text}<|im_end|>\n<embedding>
image: <|im_start|>user\n<__media__><|im_end|>\n<embedding>

<__media__> is the llama.cpp mtmd media marker; mtmd adds the vision start/end tokens. <embedding> (id 248077) is a control token and must be parsed as special.

Conversion

Made with llama.cpp 5343f45 (convert_hf_to_gguf.py, llama-quantize) from the official safetensors, with two adjustments:

1. WeMM's tokenizer.json adds a normalizer (strips a trailing \n) and a post-processor (appends <embedding>). Both change the converter's pre-tokenizer hash. The conversion used a copy with normalizer set to {"type": "NFC"} and post_processor set to null. Vocabulary and merges are unchanged; the prompt above adds <embedding> explicitly.
2. --no-mtp: the config declares an MTP layer that the weights do not contain.

Text model: f16 → Q4_K_M. Vision projector: f16.

Evaluation

Flickr30k 1k test (1,000 images, 5,000 captions), images resized to a maximum side of 384 px, through imgsearch's llama.cpp pipeline:

Model	Quant	T2I R@1	T2I R@5	I2T R@1	ms/image (M2 Max)	ms/image (Radeon 780M)
Qwen3-VL-Embedding-2B	Q6_K	80.8	94.7	94.3	242	–
Qwen3-VL-Embedding-8B	Q4_K_M	81.1	95.0	94.5	799	1095
WeMM-Embedding-2B	Q6_K	84.2	96.6	94.7	202	–
WeMM-Embedding-2B (this repo)	Q4_K_M	84.1	96.5	95.0	236	421

Against the transformers reference, text embeddings reach cosine ≈0.99 at Q4_K_M (0.9999 at Q8_0). Image embeddings reach 0.95–0.97 with identical pixels; this gap comes from llama.cpp's vision implementation, not from quantization.