jersobh/epochdb-embed

EpochDB Embed

Fine-tuned multilingual embedding model for EpochDB retrieval, based on intfloat/multilingual-e5-small.

Optimized for ranking current entity facts above outdated ones (state supersession), with English, Portuguese, and German support.

Model details

Item	Value
Base	intfloat/multilingual-e5-small
Dimensionality	384
Max sequence length	512
Similarity	Cosine (normalize embeddings)
Framework	Sentence Transformers

Prefixes (required for E5)

• Queries: query: ...
• Documents / passages: passage: ...

Usage (Sentence Transformers)

from sentence_transformers import SentenceTransformer

model = SentenceTransformer("jersobh/epochdb-embed")

query = "query: Where does Lukas work and live now?"
docs = [
    "passage: Fact: Lukas is currently working at Sixt and living in Berlin.",
    "passage: Outdated record: Lukas used to work at Hive Consulting in Lisboa.",
]

q = model.encode(query, normalize_embeddings=True)
d = model.encode(docs, normalize_embeddings=True)
scores = q @ d.T
print(list(zip(docs, scores)))

Training data (summary)

Mixed triplets:

1. MS MARCO-style retrieval slices (when available)
2. Natural Questions-style slices (when available)
3. Synthetic entity state-supersession hard negatives (EN + PT + DE)

Loss: MultipleNegativesRankingLoss
~1 epoch, batch 16 x grad accum 4, lr 2e-5, bf16

GGUF / Ollama (experimental)

GGUF may be in this repo (e.g. epochdb-embed.q8_0.gguf) and as Ollama tag jersobh/epochdb-embed.

Warning: GGUF conversion used a local llama.cpp pre-tokenizer patch. Validate against SentenceTransformers before production. HF SentenceTransformers is the source of truth.

Limitations

• Synthetic hard negatives are template-like
• Keep E5 query: / passage: prefixes
• GGUF/Ollama path is best-effort

Intended use

Semantic retrieval for EpochDB (entity-centric facts, multilingual queries), preferring up-to-date state over historical records.