Daecore/embeddinggemma-300m-memory-ft-v1

EmbeddingGemma-300M memory retriever v1

Summary

This model is the dense lane of a local memory-retrieval pipeline. It is a fine-tune of google/embeddinggemma-300m for finding passages of working notes and documentation that answer a natural-language question, trained on 116,101 graded query-passage pairs.

On a fully judged 376-query development panel it reached Hit@1 0.657 against 0.598 for the conservative first fit, a paired gain of 0.059 with a family-clustered 95% interval of [+0.014, +0.104], and Recall@50 0.547 against 0.516 [+0.005, +0.058]. In the protected promotion read, as the first stage of the fused pipeline, it lifted useful Hit@1 from 0.484 to 0.574 over the previous production retriever on 895 answerable queries.

Every relevance grade behind these numbers is a frontier-model judgment under a frozen two-plus-one protocol, and the corpus is mostly generated documents; the limitations section says what that does and does not license.

Model details

Item	Value
Architecture	EmbeddingGemma-300M, mean pooling, dense projection head, merged attention LoRA (rank 16, alpha 32)
Embedding widths	768 (qualified), 512, 256, 128 (Matryoshka-trained, storage-only)
Query prefix	task: search result | query:
Passage prefix	title: none | text:
Sequence limits	128 query tokens, 1,024 passage tokens
Serving precision	FP32 through ONNX Runtime, the qualified realization; no FP16 realization is qualified
Deployment state	protected gate passed; package qualified; not yet published or activated

Intended use

Semantic candidate discovery inside a hybrid first stage: this model and BM25 each retrieve, reciprocal-rank fusion (rank constant 60) merges them, and a fixed pool of 50 passages goes to a cross-encoder reranker. The model is tuned for rich, question-shaped memory queries. Terse, exact, or pasted queries are the lexical lane's job; the model was neither trained nor selected on them alone.

Training data

Item	Value
Queries in gradients	2,612 answerable queries plus the 261-query tuning fold
Unique positive pairs (grades 2 and 3)	116,101, each seen exactly once
Unique scheduled negatives (grade 0)	53,645
Repeated supervised pairs	0
Protected queries or families in gradients	0

Queries were authored by language models from document situations without seeing answers or candidates. Grades run 0 to 3; grade 3 positives carry 1.5 times the weight of grade 2. Same-family co-batched positives are masked so a related passage is never punished as a false negative, and sparse "needle" queries receive one direct positive-versus-hard-negative lesson.

Recipe

Effective contrastive batch 32, learning rate 5e-5 with 10% warmup and cosine decay, FP16 with dynamic loss scaling, 3,685 optimizer updates over one coverage epoch, Matryoshka objectives at 768/512/256/128 with weights 1, 0.25, 0.125, and 0.0625, temperature 0.05. The adapter was merged into the base; merging moved probe embeddings by at most 1.4e-7 and a save-and-reload by 0.

Evaluation

Development panel. 453 queries (376 answerable, 77 with no useful passage in any judged pool), 104 query families, 75,261 grades, zero unjudged passages through rank 50 for every compared system. Dense-only, no reranker.

Model	Hit@1	Hit@5	Hit@10	Hit@20	Hit@50	Recall@50
conservative first fit (G0)	0.5984	0.7766	0.8404	0.8910	0.9654	0.5164
this model, batch 32	0.6569	0.7926	0.8617	0.9016	0.9575	0.5474
batch-64 finalist	0.6463	0.8085	0.8590	0.9016	0.9575	0.5507

The two finalists tie within uncertainty at every cutoff; the pre-registered efficiency rule selected batch 32. Real-source queries are a thin slice (43 of 376) on which this model scored Hit@1 0.744 against 0.646 on generated worlds; the slice saturates at Hit@20 and cannot discriminate deeper.

Matryoshka widths on the same panel, same vectors truncated and re-normalized. Smaller widths cut stored bytes but not the forward pass, and their changed pools were not fully judged.

Width	Hit@1	Hit@10	Hit@50	Recall@50	Unjudged@50	Bytes per vector
768	0.6569	0.8617	0.9575	0.5474	0.000	3,072
512	0.6516	0.8484	0.9415	0.5169	0.061	2,048
256	0.6250	0.8404	0.9415	0.4812	0.131	1,024
128	0.5851	0.8112	0.9388	0.4239	0.222	512

Protected gate, first stage only. 990 sealed queries, two source-disjoint draws frozen before training, every top-50 pool of both arms fully graded. On the 895 queries answerable in the matched union, useful Hit@1 was 0.5743 against 0.4838 for the previous retriever, +0.0905 with a one-sided 95% lower bound of +0.0590 against a registered 0.02 margin. Every critical slice passed and no gate query or family appears in training.

Runtime

Serves through FastEmbed 0.8.0 and ONNX Runtime 1.24.4 with the CUDA overlay, no Torch stack. The ONNX graph reproduces the merged model: the regenerated export differs from the Torch reference by at most 9.2e-7 per vector element with identical top-1 and top-10 sets at every width. Re-measured on 2026-09-04 on an otherwise idle RTX 3060 Ti over 50 repetitions, a query embeds in 0.015 s median and 0.017 s at the 95th percentile, and a batch of 16 passages takes 0.82 s at the 95th percentile.

Limitations, ranked

1. Model-judged relevance. No human grades exist; agreement between independent model sessions was 0.858 with kappa 0.682 on a 419-query audit. Scores measure agreement with that instrument.
2. Generated corpus. 80 fictional document worlds plus one operator's own projects. Behavior on other real corpora is unmeasured. The generated worlds were chosen for their breadth of domains and document shapes.
3. One training run. No seed replicate was affordable; the batch-32 versus batch-64 tie and the 0.059 gain over G0 are single-run estimates whose intervals capture query sampling only.
4. Tuning fold in gradients. The 261-query fold that chose the recipe was added to the final fit for more data once it had saturated; the selection read cannot be repeated on it.
5. Rich queries only. Terse and exact queries were left to the lexical lane and to the reranker's training mix.
6. Per-query gate evidence lost. The gate judgments and report were destroyed in a local storage incident after the aggregates above were recorded; they cannot be re-audited.

Reproducibility

• Model tree SHA-256 ae68b190bca3a5e9ac90e45ca4c1449c823899ca593d65526a96398c0e607703; base tree 24b175205b9fc9a4ba6480a54ee8ea8c26178d52ef33752d6f82d53921e96b4d.
• Training data manifest 8c0ec3eba6b641a6dc7a95886e838f6062525c4b064eb1d2ebd8b2ebbf793326, schedule 250eb68b21aa6f9332b9f08fc69d90553321784b76200b7cef8bc4804166e8d4.
• Recipe id final-coverage-lr5e5-b32; registry manifest ../model_registry/embeddinggemma-300m-memory-ft-v1.json; experiment log ../history/README.md.
• Build stack: Transformers 4.57.6, Sentence Transformers 5.5.1, PyTorch 2.10.0; export opset 18.

License

Gemma Terms of Use and the Gemma Prohibited Use Policy apply; the distribution includes the required notice and a modification disclosure. See https://ai.google.dev/gemma/terms.