Polaris

Polaris is the search model of COSMOS, an external memory for AI assistants. It turns a question and a stored memory into vectors, so COSMOS can find the right memory by meaning.

It is multilingual-e5-small (118M parameters, 384 dimensions), fully fine-tuned on memories written for COSMOS, and exported to ONNX to run on CPU with fastembed.

Usage

Polaris expects the e5 prefixes: query: before a search, passage: before a stored memory. Without them, scores collapse.

from fastembed import TextEmbedding
from fastembed.common.model_description import ModelSource, PoolingType

TextEmbedding.add_custom_model(
    model="liamlitle/polaris", pooling=PoolingType.MEAN, normalization=True,
    sources=ModelSource(hf="liamlitle/polaris"), dim=384, model_file="model.onnx",
)
model = TextEmbedding("liamlitle/polaris")
q = next(model.embed(["query: Where do I live now?"]))
m = next(model.embed(["passage: I moved to Ghent in March."]))

polaris.json holds the prefixes and the threshold COSMOS uses to recognize two versions of the same fact (cosine ≥ 0.54).

Training

• Data: 7,040 memories written for this purpose in 12 languages, on everyday and work themes. Each memory comes with three ways to ask for it later, a near-miss memory that does not answer the question, and, for about half of them, a newer version of the fact.
• Objective: MultipleNegativesRankingLoss with the near-miss as hard negative. Both versions of a fact count as right answers; COSMOS ranks the newest first by date.
• Run: 26,029 training triplets, 2 epochs, batch 64, learning rate 2e-5, on 2 Tesla T4 GPUs (about 10 minutes). 6% of the themes were held out.
• The names and values of the COSMOS benchmark were banned from the training data.

Results

COSMOS retrieval benchmark, search only (no LLM): about 200 stored memories, 3 random draws, about 125 questions each.

Measure	MiniLM-L12 (previous)	e5-small (not fine-tuned)	Polaris
Right memory ranked 1st	85.6%	87.3%	85.3%
Right memory in the top 3	97.6%	96.3%	98.3%
MRR	0.914	0.920	0.915
"Not in memory" recognized	84.6%	84.8%	92.0%
Newest version ranked first	91.7%	91.7%	86.1%

Limits

• Small benchmark: gaps of 1 to 2 points are within chance.
• The version-detection settings were tuned on the same benchmark.
• The training memories and questions were written by an AI, not collected from real users.