join3r/stem-gte-memory

Stem GTE Memory

A custom GTE reranker fine-tuned for Stem's personal-memory retrieval, with training queries in Czech, Slovak, German, and English. It scores how relevant a stored memory is to a user's message. Stem also uses the same weights for skill selection with a separately calibrated threshold.

This is the selected epoch2 q8 ONNX release, model ID gte-memory-20260905-epoch2. The five inference files total 358,119,455 bytes (341.53 MiB); Stem's dropdown rounds this to approximately 342 MB. The quantized graph alone is 341,032,790 bytes.

In Stem's fresh synthetic memory benchmark, this checkpoint improved recall from 43.1% to 64.7% and reduced irrelevant selections from 166 to 87, compared with the configured Qwen3 Reranker 0.6B baseline. A separate local timing experiment measured 320 ms vs 3,116 ms median for the benchmark pipeline, approximately 9.74× faster. These are scoped benchmark results, not universal model rankings or end-to-end chat speed guarantees.

Intended use

Rerank a shortlist of memory facts, or skill descriptions, before an assistant responds. Training and the reported multilingual evaluations used English stored facts or descriptions with multilingual user requests. Broader language coverage inherited from the base model has not been established by this fine-tune's evaluation.

This is an experimental, opt-in release. Publishing the files does not enable automatic downloading in existing Stem versions; that app integration is separate.

Files and integrity

• onnx/model_quantized.onnx: selected q8 inference graph, ONNX opset 17.
• config.json, tokenizer.json, tokenizer_config.json, special_tokens_map.json: the exact measured inference configuration and tokenizer.
• stem-model.json: immutable model identity, upstream revisions, per-file SHA-256 hashes and sizes, and Stem thresholds.
• example.mjs: local Node.js inference example using Transformers.js 3.8.1.
• LICENSE and NOTICE: licensing and modification notices.

Pin a Hub commit revision when downloading, then verify all five inference files against stem-model.json. Hashes are also pinned in Stem's runtime. No training datasets, chat transcripts, benchmark examples, FP32 weights, or training checkpoints are included.

config.json retains an inert temporary export path in _name_or_path to preserve its measured hash. It does not need to exist. Its upstream auto_map metadata is likewise preserved; the supported ONNX path below does not execute remote Python code.

Inference contract

Use Transformers.js 3.8.1 PreTrainedModel, not its auto sequence-classification loader: this export retains the upstream custom model_type: new. This repository contains ONNX weights, not a PyTorch checkpoint.

1. Tokenize a (query, document) text pair with the supplied tokenizer.
2. Score one pair per forward pass, sequentially. Do not silently truncate. Stem rejects pairs longer than 8,192 tokens, including special tokens; long-context quality has not been validated here.
3. Read the single finite raw logit. Higher means more relevant. Do not apply sigmoid or reinterpret it as a probability.

Install @huggingface/transformers@3.8.1 in a Node.js project, download this repository's files to a local folder, and run:

node example.mjs /absolute/path/to/downloaded/model

For Stem's measured configuration:

Purpose	Raw-logit threshold
Memory facts	-1.6370911598205566
Skill selection	-3.9743599891662598
Additional margin for sensitive facts	+2

These thresholds belong to this exact graph, tokenizer, and pipeline. They are not interchangeable or universal cutoffs. Stem uses a Qwen3 Embedding 0.6B shortlist, up to 24 fact candidates and at most 16 relevant selected facts (subject to a smaller user setting). Pinned facts are handled separately.

For fact retrieval with conversational context, Stem takes at most two preceding eligible user messages in chronological order, clipping each to 400 Unicode code points, and formats the query as:

Earlier messages from the user in this conversation:
- <earlier user message>
- <later user message>

Current message: <current user message>

With no eligible history, use the current message alone. Skill selection uses the current message alone and its own threshold.

Training

Base: Alibaba-NLP/gte-multilingual-reranker-base, revision 8215cf04918ba6f7b6a62bb44238ce2953d8831c. Upstream implementation: Alibaba-NLP/new-impl, revision 40ced75c3017eb27626c9d4ea981bde21a2662f4.

Fine-tuning used 23,684 binary pairs from 88 fictional training profiles, including 797 relevant pairs. Query-language pair counts: Czech 5,905; Slovak 6,051; German 5,963; English 5,765. Borderline examples were excluded. Real personal-memory benchmarks were not used for training and are not published.

The encoder and classification head were trained (113,920,513 trainable parameters); token embeddings were frozen. AdamW used learning rate 1e-5, weight decay 0.01, batch size 16, 10% linear warmup, gradient clipping 1.0 and BCE positive weight 3. The maximum tokenized training pair length was 94; no training examples were truncated. Three epochs were planned; epoch2 was selected after q8 export using development data. Feasible settings had to meet the configured Qwen baseline recall overall and per language, then minimize 3 * false negatives + false positives.

Export used PyTorch 2.8.0 and ONNX Runtime 1.22.1. Dynamic weight quantization used QInt8, per-channel quantization, reduced range, symmetric weights and constant-matrix-only MatMul quantization. Four synthetic export probes had maximum FP32/PyTorch error 0.0000019074 and q8/FP32 error 0.065004; these probes do not establish broad numerical parity.

Evaluation and limitations

The fresh memory test contained 192 requests from 16 fictional profiles, 48 in each language. Its references were synthetic author/reviewer judgments, with disagreements treated as borderline, not human gold. Labels covered the reviewed union of candidate pools, so facts missed by both retrieval paths remained unjudged.

Request language	Configured Qwen recall	Stem GTE recall
Czech	19.4%	58.1%
Slovak	10.3%	53.8%
German	48.7%	64.1%
English	74.1%	75.9%

Two previously seen real-data regression sets showed small recall changes (15.4% to 19.2%, and 23.5% to 25.7%) and fewer irrelevant selections. Their recall-difference intervals included zero; they do not establish a large real-world recall gain. No real-data examples are distributed.

The timing result above measured 64 warm observations per model on a local Mac. Each observation embedded a contextual query, sequentially scored 24 candidates and applied the gate. Candidate retrieval, database work, IPC, model loading and answer generation were excluded. Repeated observations were correlated. This is not a promise about complete response latency, other hardware, or application-wide memory consumption.

Linux validation found numerical differences from macOS despite identical tokenized inputs (maximum absolute logit difference 0.645472 on 16 probes). With the frozen fact threshold, Linux selected 106 useful and 84 irrelevant facts on the same fresh synthetic set, versus macOS GTE's 108 and 87. An English-only same-server rerun selected 42 useful facts with GTE versus 39 with Qwen; other languages' Qwen baselines were not rerun on Linux. Validate your target runtime; do not assume bitwise portability.

Sensitive-fact selection remains a limitation: in the fresh macOS test, GTE selected 3 irrelevant sensitive facts versus Qwen's 1 while finding more useful sensitive facts. The inherited +2 margin is not equally calibrated across models.

A separate fresh skill test found the correct skill on 39/48 positive requests versus Qwen's 33/48, with false loads on 11/48 negative requests versus 12/48. GTE still missed the predeclared 90% correct-loading target (81.25%). Both models found the correct skill on only 15/24 concise conversational positives. The weights were not trained again for this skill evaluation; only the threshold was calibrated beforehand.

These experiments do not establish improved final answers, long-input accuracy, performance outside the four training languages, or universal superiority over other rerankers. The model should remain a selectable option while broader evaluation continues.

License and attribution

Released under Apache License 2.0, following the upstream model's declared license. See LICENSE and NOTICE. Stem modifications comprise memory-relevance fine-tuning, ONNX export, quantization and task-specific calibration. This release is a Stem adaptation, not an official Alibaba release.