SciEmbed-BASE

Signal A only (citation edges), 7M pairs, 3 epochs. The citation-edge baseline that isolates what Signal B adds.

A 149M-parameter ModernBERT-base scientific document embedder trained with citation-context sentences as the primary contrastive signal. Part of the SciEmbed release (paper under double-blind review; author info omitted).

Usage

from sentence_transformers import SentenceTransformer

model = SentenceTransformer("anon-nlp/sciembed-base")
emb = model.encode(["citation-context supervision for scientific embeddings"],
                   normalize_embeddings=True)

Context length: 512 tokens
Pooling: mean · Output dim: 768 (Matryoshka-truncatable to 512/256/128)
License: MIT

SciRepEval (4-category macro)

Classif.	Regr.	Prox.	Search	Overall
75.3	26.8	80.2	82.2	66.1 ± 0.09

Citation

See the repository README. Paper: SciEmbed: Citation-Context Supervision for Scientific Document Embeddings (under review).

Downloads last month: 29

Safetensors

Model size

0.1B params

Tensor type

F32

Model tree for anon-nlp/sciembed-base

Base model

answerdotai/ModernBERT-base

Finetuned

(1349)

this model