NYSgpt/biomed-encoder

BioMed Encoder

Dense retrieval over biomedical scientific literature (433,449 papers).

BioMed Collection | BioMed Reranker

• ✅ Beats MedCPT on concept search: R@10 0.524 vs 0.457 (+15%), p = 1e-6 — with 2,500× less training data.
• ✅ +83% over its stock base, +49% over allenai/specter2_base on the same queries.
• ✅ No labels, no click logs: training pairs come from the papers' own MeSH descriptors and author keywords.
• ✅ Drop-in: standard sentence-transformers bi-encoder, 1024-d cosine vectors, TEI-servable, MIT-licensed.

ncbi/MedCPT — the National Library of Medicine's biomedical retriever — learned search from ~255 million real PubMed click pairs. This model learned it from 100,000 pairs — and on the same frozen benchmark it beats MedCPT on concept search.

First stage of a two-stage stack — pair it with the BioMed Reranker for best results.

---

Details

Property	biomed-encoder
Type	Dense bi-encoder (single vector)
Total parameters	~568M
Backbone	BAAI/bge-m3
Output	1024-d normalized vector
Similarity	Cosine
Sequence length	128 query / 384 passage
Training signal	100k mined query→paper pairs
Built for	Search (query→document)
MeSH concept R@10	0.524
License	MIT

---

Performance

MeSH concept queries — expert subject headings, the professional-search case. Every arm is scored on the same 1,241 held-out queries, retrieved against the same 433,449 documents, with the same metric code — on a benchmark frozen before any training and split by paper (BioMed Eval — queries, gold keys and per-query results are public). Baselines are measured here, not quoted from their papers, each model with the similarity function its authors published it for.

Rank	Model	Params	R@1	R@10	nDCG@10
—	RRF(FTS + this model) (the production arm)		0.272	0.533	0.394
1	biomed-encoder (ours)	568M	0.269	0.524	0.388
2	ncbi/MedCPT	2×110M	0.235	0.457	0.337
3	allenai/specter2_base	110M	0.153	0.351	0.247
4	BAAI/bge-m3 (stock base)	568M	0.118	0.286	0.194
5	Postgres FTS	—	0.047	0.081	0.063

The win over MedCPT is a paired exact McNemar result at p = 1.0 × 10⁻⁶ (184 queries gained vs 101 lost), bootstrap 95% CI on ΔR@10 [+0.040, +0.093], 5,000 resamples.

---

Training

Objective	in-batch contrastive, dense only (--unified_finetuning False)
Trainer	FlagEmbedding finetune.embedder.encoder_only.m3
Pairs	100,000 — MESH 55,000 · AUTHORKW 40,000 · EXTRACT 5,000
Negatives / pair	8 · group size 8
Batch / epochs / steps	8 / 1 / 12,500
Precision	bf16
Hardware / wall-clock	1× NVIDIA L40S (g6e.2xlarge) · 2 h 22 m

Pairs are mined from the corpus's own structure: MeSH descriptors and author keywords as queries, the paper they annotate as the positive. No benchmark gold paper appears anywhere in the training pairs.

---

How to run

from sentence_transformers import SentenceTransformer

model = SentenceTransformer("NYSgpt/biomed-encoder")

query = model.encode(["gut microbiome depression"], normalize_embeddings=True)
docs = model.encode(
    ["Interplay of Human Metabolome and Gut Microbiome in Major Depression ..."],
    normalize_embeddings=True,
)
print(query @ docs.T)

Serves cleanly on Hugging Face TEI. sentence_bert_config.json carries max_seq_length 384, matching training — do not raise it at serve time without re-embedding the corpus.

For best quality, add the second stage: re-scoring this model's top-50 with biomed-reranker lifts R@10 from 0.524 to 0.549 and R@1 from 0.269 to 0.306.

---

📬 Contact

Questions, results, or a use case to share? Open a discussion in the Community tab.

Citation

@misc{cshlcpt2026,
  title  = {CSHL-CPT: a retrieval stack for the complete bioRxiv and medRxiv corpus},
  author = {NYSgpt},
  year   = {2026},
  url    = {https://huggingface.co/NYSgpt/biomed-encoder}
}