flowgraph-a2-student-l12

The embedding model used by FlowGraph — a local-first desktop app that gives AI tools persistent, graph-shaped memory stored on the user's own machine.

These are FlowGraph's own trained weights: a MiniLM-L12 student distilled from a permissively-licensed offline teacher. No third-party model is redistributed here. The model runs locally and in-process via ONNX Runtime — no API key, no network call at inference time.

What's in this repo

A single packaged bundle, embedder-onnx-bundle-a2-student-l12.tar.gz (~124 MB), which FlowGraph's installer downloads by pinned digest. Inside, at the archive root:

file	sha256
model.onnx	d99dbe939d43fd691d8ad36665f5f74592930d9d41ec00c47f44e32f0899bb27
vocab.txt	26e5c70d53771ba1a86b01f21baed1bf6f401236bcc15fd2cb73f0a0ea5aba66
config.json	be86edead4f536af99471225bfb498006c897aa2db9c754cc36a97d9c7d43783
tokenizer.json, tokenizer_config.json	(HuggingFace tokenizer artifacts; unused by FlowGraph's in-tree WordPiece tokenizer, which reads vocab.txt)

Bundle identity (specVersion, a merkle-sha256 over model.onnx + vocab.txt + config.json):

96e68b60d45a33a2f3c5a53d37b01f17c36b1378819cd8f3a9f6ad5d8ebee195

Archive sha256: 0946c46dba4c5682e5567b53563a5f4f493abc2199677482ac80090720afd11d

Architecture

MiniLM-L12 (12 layers, hidden 384, 12 heads, ~34M params) → mean pooling → Dense(384 → 768, no bias, identity activation) → L2 normalize. The exported ONNX bakes the whole pipeline in: it takes input_ids + attention_mask and emits a single L2-normalized 768-d sentence_embedding per row. There is no token_type_ids input.

• dimensions: 768
• max sequence length: 256
• text prefix: search_document: — prepended to both passages and queries (see below)
• tokenizer: BERT WordPiece, uncased, accent-stripping

Usage

import numpy as np, onnxruntime as ort
from tokenizers import Tokenizer

tok = Tokenizer.from_file("tokenizer.json"); tok.enable_truncation(max_length=256)
sess = ort.InferenceSession("model.onnx", providers=["CPUExecutionProvider"])

def embed(texts):
    enc = [tok.encode("search_document: " + t).ids for t in texts]   # NOTE: same prefix for queries
    L = max(len(e) for e in enc)
    ids = np.zeros((len(enc), L), dtype=np.int64)
    mask = np.zeros((len(enc), L), dtype=np.int64)
    for i, e in enumerate(enc):
        ids[i, :len(e)] = e; mask[i, :len(e)] = 1
    return sess.run(None, {"input_ids": ids, "attention_mask": mask})[0]   # already unit-normalized

The prefix is symmetric. Unlike nomic-class models, this student was distilled on a single search_document: regime, so queries take the same prefix as passages. Using search_query: puts the input out of distribution.

Training

Distilled from nomic-ai/nomic-embed-text-v1.5 at revision e9b6763023c676ca8431644204f50c2b100d9aab (Apache-2.0), used strictly as an offline teacher — its weights are not redistributed.

loss	pointwise MSE to the teacher's L2-normalized embedding + relational in-batch pairwise-similarity MSE (match the teacher's rankings, not just its coordinates)
corpus	wikimedia/wikipedia, config 20231101.simple, split train; texts with len > 100, truncated to the first 512 chars, first 150,000 in dataset order
epochs / batch / lr / warmup	4 / 128 / 2e-4 / 100
precision	AMP (fp16), single GPU
max seq length	256
student init	sentence-transformers/all-MiniLM-L12-v2 + the Dense projection head + Normalize

The relational term is what made this work: plain MSE-to-embedding plateaued around 59% on an internal doc-level proxy regardless of student size (23M / 34M / 110M all landed in the same band), while adding the in-batch pairwise-similarity term lifted the 34M student to ~71% — about 95% of the teacher at roughly a quarter of the parameters.

Reproducibility. No RNG seed was set during training, so bit-identical retraining is not possible by construction. The contract is: the released checkpoint is pinned by hash (above) and the procedure is documented; reproduction means equivalent under this procedure, not bit-identical.

Export

torch.onnx.export (legacy exporter, dynamo=False), opset 17, do_constant_folding=True, dynamic axes {0: batch, 1: seq} on the inputs. Attention is forced to eager at export time. (For the record: an earlier SDPA-traced export produces numerically identical vectors — cosine 1.00000 and 100% top-10 neighbour overlap across a probe set — so the attention implementation is pinned for procedure fidelity, not because it changes the vector space.)

Faithfulness was verified against the PyTorch reference on 200 real corpus texts (including 73 short, label-like strings): min cosine 1.00000, mean 1.00000, top-10 neighbour overlap 100.0%.

Evaluation

Measured inside FlowGraph's retrieval benchmark at node level, with the graph held constant and only the embedder swapped (so the numbers isolate the embedder rather than the ingest), 200 labeled queries per dataset, k=10:

	ArguAna (1000 distractors)	HotpotQA
this model — vector recall@10	38.5%	60.5%
FlowGraph's previous static embedder	30.5%	31.0%
teacher (nomic-embed-text-v1.5)	48.5%	65.3%
this model — hybrid nDCG@10	0.330	0.611
teacher — hybrid nDCG@10	0.340	0.612

On raw vector recall the student reaches 79–93% of its teacher. On the fused hybrid channel the app actually serves (reciprocal-rank fusion of vector + BM25 + graph walk) it reaches 97–100% of the teacher, and slightly exceeds it on HotpotQA MRR. Note that these are node-level scores over an LLM-extracted knowledge graph, not standard BEIR document retrieval numbers, so they are not comparable to published BEIR leaderboards.

Inference is CPU-cheap: roughly 25 ms/text single-threaded on a 2-core container for short inputs.

Intended use & limitations

Built for retrieval over a personal knowledge graph: short node labels and document chunks, English, ≤256 tokens. It is a distilled student — on raw semantic recall the teacher is still better, and it inherits the teacher's and the Simple-English-Wikipedia corpus's biases. Not evaluated for classification, clustering, or non-English text.

License

Apache-2.0. The weights are FlowGraph's own; the teacher was used offline under its own Apache-2.0 terms and is not redistributed here.