ThakiCloud/eg2-tiny-query-commerce-ko

🇰🇷 한국어 카드: MODEL_CARD.ko.md

EG2 Tiny Query Encoder: Commerce (Korean + English)

Two small INT8 query encoders that write Korean and English shopping queries into the image embedding space of google/embeddinggemma-2. Keep your EmbeddingGemma 2 product-image index and search it without re-indexing.

What it is	Two TinyCLIP Text-3M text towers + 256→768 linear layers, distilled from EmbeddingGemma 2. ko/ = K4 (Korean-trained), en/ = E0 (English product search)
Default routing	a query containing Hangul goes to K4, any other query to E0. route="ko" or route="en" forces one encoder
Input → output	search text → one L2-normalized 768-d vector
Compare with	product-image vectors produced by google/embeddinggemma-2 (dot product)
Weights	row-wise INT8 (one fp16 scale per row), dequantized to float32 at load
Weights + tokenizer size	32,971,443 bytes = two model.safetensors (15,368,930 each) + one shared tokenizer.json 2,224,041 + four configs 9,542
Max query length	K4: 77 tokens; a longer query keeps its first 57 and last 20 tokens. E0: 64 tokens, plain truncation. No query prefix
Not included	the image encoder (you need EmbeddingGemma 2 to build image vectors), any keyword index, any catalog
Training	one run each, seed 0
Versions	K4 (current, 2026-10-19). Earlier Korean encoders stay available as revisions: K3 as k3, K2 as k2, e.g. snapshot_download("ThakiCloud/eg2-tiny-query-commerce-ko", revision="k3")

Quick start

import sys
import numpy as np
from huggingface_hub import snapshot_download

repo = snapshot_download("ThakiCloud/eg2-tiny-query-commerce-ko")
sys.path.insert(0, repo)
from commerce_query import CommerceQueryEncoder

enc = CommerceQueryEncoder.from_pretrained(repo)  # Hangul -> K4, otherwise E0
q = enc.encode(["무선 블루투스 이어폰 화이트", "stainless steel water bottle"])
print(q.shape)  # (2, 768)

# Your existing index: (N, 768) L2-normalized EmbeddingGemma 2 product-image vectors.
image_vectors = np.load("image_vectors.npy")
top10 = np.argsort(-(q @ image_vectors.T), axis=1)[:, :10]
print(top10[:, :3])

Requirements: torch, transformers, safetensors, huggingface_hub, numpy. CPU is enough.

No index yet? example_search.py builds one with EmbeddingGemma 2 and searches it. It can also fuse the model's top 10 with a BM25 keyword index over your own product titles (reciprocal rank fusion, k = 60). Building the index needs pip install "sentence-transformers[image]>=6.1.0".

python example_search.py build  --images ./products --index index.npz
python example_search.py search --index index.npz --query "여성 겨울 롱패딩 블랙" --query "usb-c charging cable 2m"
python example_search.py search --index index.npz --catalog catalog.jsonl --query "나이키 러닝화"   # model + BM25

catalog.jsonl has one line per indexed image: {"path": "<same path as in the index>", "text": "<title, brand, attributes>"}.

Image vectors must come from google/embeddinggemma-2 encoding the image alone ({"image": PIL.Image} in sentence-transformers), L2-normalized.

What was measured

In short (K4 vs K3, development sets, R@10):

• Synthetic descriptive queries 0.489 → 0.652; Korean product names 0.665 → 0.711.
• The synthetic set comes from the same generator as the K4 training queries, so improvement on real user queries is not yet confirmed.
• Very short product names (35 queries of ≤ 16 tokens) did not improve, and K-Fashion (0.379) is still below K1 (0.403). For English, the default routing still uses E0 and its numbers are unchanged; forcing K4 with route="ko" is reported separately below.

All numbers are from development sets. Three of them were also used to choose the K4 checkpoint: Q-TITLE-ko, K-Fashion and the synthetic descriptive-query set (the mean of Q-TITLE-ko R@10, K-Fashion P@1 and synthetic-query R@10 after each epoch). The last epoch (3 of 3) won, so there was no early stop, but these numbers are still not independent of that choice. kobench was not used for the choice this time. Nothing here comes from a held-out test; the sealed Commerce test sets were not opened. Everything is scored on the INT8 files of each version. The image pool is 27,309 ABO development products (Amazon Berkeley Objects). Intervals are 95% cluster-bootstrap intervals over the paired per-query difference; they capture query sampling, not training-seed variation.

Model alone, R@10

The K3 and K4 columns send every query to that encoder; the K4 − K3 comparison below uses only those two columns. The routed column is the default call (Hangul → K4, otherwise E0) and is reported separately.

Query set	n	E0 only (route="en")	K3 (previous, revision="k3")	K4 only (route="ko")	Automatic routing (default)	Teacher (1.5 GB)
Synthetic descriptive queries (model-written, see below)	4,266	0.091	0.489	0.652	0.652	0.663
Q-TITLE-ko: Korean product names (ABO Korean listings)	889	0.368	0.665	0.711	0.710	0.682
kobench: Korean template queries over ABO attributes	173	0.104	0.358	0.462	0.462	0.491
Q-TITLE: English product names	5,000	0.805	0.808	0.811	0.805	0.773
Q-KW: English listing keywords	3,064	0.305	0.286	0.276	0.305	0.276
ESCI: real English shopping queries (Exact matches)	908	0.422	0.522	0.532	0.422	0.626

K-Fashion (Korean fashion attributes, in-distribution, P@1, n = 626, pool 20,000): E0 0.005 · earlier K1 0.403 · K2 0.289 · K3 0.372 · K4 0.379 · teacher 0.176.

• Synthetic descriptive queries, K4 − K3: +0.163 [+0.141, +0.186] R@10 (227 product-type clusters); R@1 0.154 → 0.236. This set is written by the same generator as the K4 training queries, about 1,229 development products that share no product with training. It shows K4 learned that kind of query; it is not evidence about human-written queries.
• Korean product names, K4 − K3: +0.046 [+0.028, +0.066] R@10 (150 clusters); R@1 0.254 → 0.278.
• kobench: K4 − K3 = +0.104, interval above zero, but with 24 product-type clusters (we require 30) it stays uncertain.
• K-Fashion: K4 − K3 = +0.006 [−0.016, +0.032] P@1, so no clear change; still below the earlier K1 (0.403).
• English, K4 − K3 with route="ko": product names +0.003 [−0.004, +0.011], real shopping queries +0.010 [−0.011, +0.030], listing keywords −0.010 [−0.020, −0.001], a small but real drop. The default routing sends English to E0, so routed English numbers did not change.
• Korean product names against E0: K4 − E0 = +0.343 [+0.306, +0.384] R@10.

Synthetic descriptive queries by style, R@10 (same set; intervals are query-level bootstrap)

Style	What the query looks like	n	K3	K4	K4 − K3
natural	a sentence describing the item without its name	909	0.339	0.629	+0.290 [+0.256, +0.325]
para	a reworded description	913	0.346	0.632	+0.286 [+0.253, +0.320]
cond	item + one or two required attributes	508	0.557	0.648	+0.091 [+0.059, +0.120]
short	a few Korean words	897	0.571	0.639	+0.068 [+0.046, +0.091]
mix	Korean with a brand or model name	1,039	0.642	0.702	+0.060 [+0.040, +0.079]
all queries with no Latin letters		2,602	0.439	0.648	+0.208 [+0.190, +0.227]

Very short product-name queries did not improve: on the 35 Q-TITLE-ko queries of 16 tokens or fewer, R@10 went from 0.43 (K3) to 0.40 (K4). That is one query, so read it as "no gain", not as a measured loss.

Fused with BM25, reference only (reciprocal rank fusion of the two top-10 lists, k = 60; BM25 over the product title text, Korean + English names for Korean queries). This is not the recommended way to search product names; it is here so you can see what each part adds.

Query set	BM25 R@1	BM25 R@10	BM25 + K4 only R@1	BM25 + K4 only R@10
Synthetic descriptive queries	0.174	0.290	0.185	0.634
Q-TITLE-ko	0.911	0.993	0.724	0.992
kobench	0.139	0.405	0.243	0.532
ESCI	0.147	0.414	0.152	0.566

• Synthetic descriptive queries share few words with the titles, so BM25 alone finds little (R@10 0.290). Fusion (0.634) stays below K4 alone (0.652).
• Q-TITLE-ko: fusion lowers R@1 sharply (0.911 → 0.724) and does not raise R@10. The query is the product's own Korean name and that name is in the BM25 index, so keyword search is close to perfect. If your catalog already has the Korean text, the model adds nothing on this kind of query.
• kobench: +0.127 [+0.024, +0.241] R@10 over BM25, uncertain with 24 clusters.
• ESCI (English): fusion with K4 adds +0.152 [+0.110, +0.191] R@10 over BM25.
• What the model is for: searching product images with Korean queries, including descriptions that do not use the product's name, when the catalog lacks matching Korean text; and widening a keyword search's top 10. It is not a first-result ranker.

Install check (not a benchmark): in a clean environment we downloaded the release files, built an EmbeddingGemma 2 index of 200 ABO development products (one per product cluster) and searched it with their Korean names using the default routing. 194 of the 200 names contain Hangul and went to K4; the other 6 Korean-listing names are English text (for example "AmazonBasics Hanging Planter, Square") and went to E0. Result: R@1 0.68, R@10 0.955 (K3 gave 0.66 / 0.96 on the same check; on 200 products one hit is 0.005). With English names (all routed to E0): R@1 0.83, R@10 1.00. The pool is tiny, so these numbers only show the files load and search correctly; use the tables above for quality.

Query length: on Q-TITLE-ko, 27.8% of queries exceeded the old 64-token limit and 10.8% exceed 77. K4 keeps the first 57 and last 20 tokens of those, so trailing colour, size or quantity words are not cut off.

Video and audio indexes: measured, not recommended (2026-10-20)

We checked whether this encoder can search BF16 EmbeddingGemma 2 video and audio vectors. The index and queries were the same as for the BF16 text tower; only the query encoder changed.

Index (BF16 EG2)	Queries	BF16 text tower R@10	K4 (default routing) R@10
Video, Korean captions (AI Hub 71866 dev, 8 frames per segment)	537	0.639	0.062
Sound description, English (FSD50K eval uploader descriptions)	802	0.222	0.121 (all routed to E0)
Spoken content, Korean (AI Hub 71699 summary → speech clip)	1,000	0.880	0.047

Every difference from BF16 has its 95% interval below zero. The encoder runs, but it was distilled for image vectors only and does not carry over to video or audio despite sharing the 768-d space. Use it for image indexes; for video and audio, encode queries with EmbeddingGemma 2 itself.

Not claimed

• Results on human-written Korean shopping queries. There are none in our evaluation. The synthetic descriptive set comes from the same generator as part of the training data, Q-TITLE-ko is listing text and kobench is template-generated.
• That a particular part of the K4 data caused a particular gain. K4 added the descriptive queries, their confusion pairs and a false-negative mask together.
• Results on a held-out test set, or across training seeds (one run per version).
• First-result accuracy. The models are meant to put the right product in the top 10.
• Latency or memory on any device.
• Compatibility with quantized EmbeddingGemma 2 image vectors (not tested).

Training (K4)

K4 continues from the released K3 weights. Tokenizer, 77-token input with head + tail truncation, architecture, loss and hyperparameters are the same as K3. E0 is unchanged.

• Teacher: google/embeddinggemma-2 (Apache-2.0), BF16. Query targets use its SearchQuery prompt; image targets are its embeddings of training-product images.
• Data, sampled by fixed weights per epoch (200,000 samples × 3 epochs; actual exposures in brackets; K3 weight in parentheses):
  • ABO Korean product names, 5,234 rows, weight 0.12 [72,000] (0.12)
  • Generated Korean shopping queries used since K2, 100,052 rows, weight 0.14 [84,000] (0.33)
  • Descriptive Korean queries, 94,042 rows for 23,175 ABO training products, weight 0.20 [120,000] (new). Five styles: short, natural sentence, conditional (item + required attributes), Korean with a brand or model, and reworded description. Written by an in-house Qwen3.8-27B (NVFP4) deployment from the product's own data. A query was dropped if it used a Latin word, number, colour, material or audience word not in the product data, a subjective word such as "pretty" or "best", a brand outside the brand style, or if it nearly duplicated another query for the same product. Each row records its product, style, the attributes it relies on and the generator. These are model-written, not human queries.
  • ABO English names and keywords, 185,759 rows, weight 0.12 [72,000] (0.15)
  • ESCI English queries with an Exact training product, 2,111 rows, weight 0.05 [30,000] (0.05)
  • K-Fashion attribute queries (AI Hub 51), weight 0.20 [120,000] (0.22)
  • K-Fashion confusion pairs (built in K3), 4,018 pairs, weight 0.07 [42,000] (0.07)
  • ABO confusion pairs (built in K3), 19,603 pairs, weight 0.05 [30,000] (0.06)
  • Descriptive-query confusion pairs, 14,453 pairs, weight 0.05 [30,000] (new). For a descriptive query, the paired product is one of K3's top 30 that differs in an attribute the query states (colour 10,209, material 2,806, others fewer). Products whose value for that attribute is unknown were not used.
• False-negative mask: in the in-batch InfoNCE, other images of the same ABO product group (variants or duplicates of one listing) are not counted as negatives. About 0.02% of logits were masked.
• Only ABO training-split products and K-Fashion training images were used. The synthetic development set uses separate development products. Training text identical to a development or sealed query string was removed.
• Loss: 1 − cos(student, teacher query vector) + in-batch InfoNCE against teacher image vectors (τ = 0.05). 3 epochs, batch 512, AdamW, lr 1e-4 (encoder) / 1e-3 (linear layer), 3% warmup, seed 0, one GPU.
• Earlier versions: K3 (revision k3) continued from K2 and added the two older confusion-pair sets. K2 (revision k2) started from E0. E0: same architecture from TinyCLIP, trained on 295,831 texts: ABO English names and keywords, ABO Korean names (1.8%), Shopify product titles.
• Base: wkcn/TinyCLIP-ViT-8M-16-Text-3M-YFCC15M text tower (MIT) @ a2a8c6eaa2549ad66eb7c31b85022bf58273a26c.
• Quantization: every 2-D weight with ≥ 4,096 elements is symmetric row-wise INT8 (scale = row max / 127, fp16); other tensors fp16. All numbers were scored after this step.

Files

File	Bytes	Purpose
ko/model.safetensors	15,368,930	K4 INT8 weights + .__scale tensors
ko/config.json	197	77 tokens, head+tail truncation (20 tail tokens)
en/model.safetensors	15,368,930	E0 INT8 weights
en/config.json	141	64 tokens, plain truncation
ko/base_config.json, en/base_config.json	4,602 each	text tower settings
tokenizer.json	2,224,041	CLIP BPE tokenizer shared by both
tokenizer_config.json	37	tells AutoTokenizer to load it as CLIPTokenizer
commerce_query.py		loader, routing and encode()
example_search.py		build an EmbeddingGemma 2 image index, search it, optional BM25 fusion

Data and licenses

• Model weights and code: MIT (see LICENSE; includes the TinyCLIP notice).
• No training data is redistributed here.
• Amazon Berkeley Objects (ABO): CC BY 4.0, © Amazon. Product names, keywords and images were used for training and evaluation.
• ESCI (Amazon Shopping Queries), via tasksource/esci: Apache-2.0.
• Shopify product catalogue (E0 training): Apache-2.0.
• This research used datasets from 'The Open AI Dataset Project (AI-Hub, S. Korea)'. All data information can be accessed through 'AI-Hub (www.aihub.or.kr)'. Dataset used: 51 (K-Fashion). Only the trained model is released; the AI Hub data itself is not redistributed, as AI Hub terms require.
• The teacher, EmbeddingGemma 2, is Apache-2.0. These encoders contain none of its weights.