NYSgpt/biomed-reranker

BioMed Reranker

Cross-encoder reranking over biomedical scientific literature (433,449 papers).

BioMed Collection | BioMed Encoder

• ✅ Beats the NLM's own cross-encoder on concept search: handed ncbi/MedCPT-Cross-Encoder's own candidate pool, R@10 0.544 vs 0.525, p = 0.013.
• ✅ +14% top-1 accuracy over the encoder's raw top-50 (R@1 0.269 → 0.306), nDCG@10 +8% — a paired win, significant in both query families.
• ✅ Two-stage system beats every open baseline on concept search, and edges MedCPT's complete retriever + cross-encoder stack (0.549 vs 0.525 R@10).
• ✅ Drop-in: standard sentence-transformers CrossEncoder, 326 pairs/s on one A10G, Apache-2.0.

A 149M cross-encoder fine-tuned from Alibaba-NLP/gte-reranker-modernbert-base to re-score retrieval candidates. Takes (query, document), returns a relevance logit. Second stage over any dense or hybrid top-K.

Second stage of a two-stage stack — pair it with the BioMed Encoder for best results.

---

Details

Property	biomed-reranker
Type	Cross-encoder (pointwise reranker)
Total parameters	149M
Backbone	Alibaba-NLP/gte-reranker-modernbert-base (ModernBERT)
Output	Single relevance logit per (query, document) pair
Context	8,192 tokens native; trained and served at 256
Training signal	40k mined query→paper groups, hard negatives
Built for	Second-stage reranking of search results
Pair with	BioMed Encoder (first stage)
MeSH concept R@10	0.549 (over the encoder's top-50)
License	Apache-2.0

---

Performance

MeSH concept queries — expert subject headings, the professional-search case. Benchmark BioMed Eval, frozen before any training, split by paper; 1,241 held-out queries, candidate pools retrieved against all 433,449 papers. Paired exact McNemar and 5,000-sample bootstrap CIs throughout.

What the second stage buys — the same top-50, before and after this model:

Metric	Encoder top-50 alone	+ this reranker	Δ
R@1	0.269	0.306	+14%
R@10	0.524	0.549	+5%
nDCG@10	0.388	0.419	+8%

The full two-stage system — this reranker over the BioMed Encoder's top-50, the production configuration:

Rank	System	R@1	R@10	nDCG@10
1	BioMed Encoder + this reranker (ours)	0.306	0.549	0.419
2	ncbi/MedCPT + MedCPT-Cross-Encoder (NLM's full stack)	—	0.525	0.405
3	BioMed Encoder, retrieval only	0.269	0.524	0.388
4	allenai/specter2_base retrieval	0.153	0.351	0.247
5	Stock BAAI/bge-m3 retrieval	0.118	0.286	0.194
6	Postgres FTS (keyword search)	0.047	0.081	0.063

Head-to-head with the NLM's cross-encoder. Handed MedCPT's own candidate pool, this reranker beats ncbi/MedCPT-Cross-Encoder on concept queries — R@10 0.544 vs 0.525, p = 0.013. MedCPT-Cross-Encoder runs zero-shot on preprints here, while MedCPT's PubMed training data overlaps substantially with this corpus.

Candidate pool ceiling — R@50 of the retriever that built each pool, the most any reranker can recover:

Pool	MeSH R@50
BioMed Encoder	0.673
ncbi/MedCPT	0.624

Feed this model a better retriever and its ceiling moves with it.

---

Training

Objective	grouped cross-entropy — 1 positive vs hard negatives per group
Groups	40,000 — family mix MESH 61% / AUTHORKW 39%, matching the benchmark
Negatives	mined from the retrieval space itself, not sampled at random — a reranker trained on random negatives learns topicality, not ranking
Epochs / text budget	2 / query 600 · document 900 characters (identical to the serving path)
Hardware / wall-clock	1× NVIDIA L40S (g6e.2xlarge) · 42.3 min
Run	20260819-223601-train · 2026-08-20

All benchmark gold papers are excluded from training as queries, positives, and negatives.

---

How to run

from sentence_transformers import CrossEncoder

ce = CrossEncoder("NYSgpt/biomed-reranker", max_length=256)
ce.predict([("gut microbiome depression",
             "Interplay of Human Metabolome and Gut Microbiome in Major Depression ...")])

Retrieve a top-50 with the BioMed Encoder, re-score with this model, serve the reordered list. p50 151 ms per query over 50 candidates on one A10G — fast enough to sit behind a live search box on a single cheap GPU.

---

📬 Contact

Questions, results, or a use case to share? Open a discussion in the Community tab.

Citation

@misc{biomedreranker2026,
  title  = {BioMed Reranker: a cross-encoder for the complete bioRxiv and medRxiv corpus},
  author = {NYSgpt},
  year   = {2026},
  url    = {https://huggingface.co/NYSgpt/biomed-reranker}
}