s-m-quadri/ltr-bert-sir

LTR-BERT SIR: pool-restricted bi-encoder for MS MARCO document ranking

Fine-tuned LTR-BERT-style bi-encoder and scoring code for pool-restricted hybrid document ranking on MS MARCO.

Authors: Syed Minnatullah Quadri, Vrishali A. Chakkarwar
Affiliation: Department of Computer Science and Engineering, Government Engineering College, Aurangabad, Maharashtra 431001, India
Contact: mt24f05f001@geca.ac.in, hi@s-m-quadri.me, vachakkarwar@geca.ac.in
Model: s-m-quadri/ltr-bert-sir

Quick start (notebook)

pip install torch transformers huggingface_hub

import sys
from huggingface_hub import snapshot_download

root = snapshot_download("s-m-quadri/ltr-bert-sir")
sys.path.insert(0, root)

from inference import LTRBertSIR

ranker = LTRBertSIR.from_pretrained(root)

query = "bm25 lexical ranking information retrieval"
docs = [
    "BM25 is a bag-of-words ranking function using term frequency and inverse document frequency.",
    "The weather tomorrow will be sunny with light winds.",
]

print(ranker.score(query, docs[0]))
# 0.8685

print(ranker.rank(query, docs))
# [
#   ("BM25 is a bag-of-words ranking function ...", 0.8685),
#   ("The weather tomorrow will be sunny ...", 0.7953),
# ]

Input	query string + passage text (or a list for .rank)
Output	float, or [(text, score), ...] sorted high → low
Load	LTRBertSIR.from_pretrained(...) (Hub id or path from snapshot_download)
Long docs	ranker.score(query, text, document=True)

Scores are cosine-style (roughly in [-1, 1]); use relative order inside a candidate list. Paper pipeline pairs this model with BM25 first. First run downloads ~1.3 GB (cached). CPU is fine.

Associated publications

Syed Minnatullah Quadri and Vrishali A. Chakkarwar.
Empirical Analysis of Score Fusion Strategies under Pool-Restricted Dense Encoding for Ad Hoc Document Retrieval.
MSW Management Journal, article 4038.
Reports pool-restricted fusion (BM25, α-blend, stratified, RRF, semantic-only).

Extended experimental manuscript (full cascade evaluation):

Syed Minnatullah Quadri and Vrishali A. Chakkarwar.
Lexical First Stage, Pool-Restricted Dense Scores, and Cross-Encoder Reranking on MS MARCO Document Development.
Adds RRF3 prior, MiniLM CE cascade, and out-of-fold learned fusion; source of the full metrics table below.

System overview

End-to-end document retrieval on MS MARCO document development (5,193 topics with official qrels):

1. First stage: PySerini BM25@100 (k1=4.46, b=0.82)
2. Bi-encoder: LTR-BERT-style model (this checkpoint); dense scores only for documents in each query's BM25@100 pool
3. Fusion: min-max normalized linear blend with α = 0.85 (BM25 weight); RRF / stratified / OOF variants in the extended manuscript
4. Late stage: MiniLM cross-encoder (cross-encoder/ms-marco-MiniLM-L-6-v2) on the top-32 of the RRF3 prior

This repository releases the fine-tuned bi-encoder weights and scoring code. Lexical indexing and the cross-encoder use public tooling (cross-encoder/ms-marco-MiniLM-L-6-v2).

Model

Item	Value
Backbone	bert-base-uncased
Compression	Linear 768 → 24
Special tokens	[Q], [D]
Query length	50 (expanded + MASK pad)
Passage segment length	150 content tokens
Document score	max over segments (presentation matching)
Training	MS MARCO passage triples; BCE on pos/neg segment scores; AdamW 2e-5; 3 epochs
Checkpoint file	checkpoints/ltr_bert_final.pt (~1.3 GB)

Architecture follows Wang et al. (LTR-BERT presentation learning on short text for long-document retrieval), with pool-restricted encoding as reported in the associated papers.

Results (MS MARCO document development)

Macro-averages over 5,193 topics. All methods operate on the BM25@100 pool. Values match the extended manuscript evaluation table (same aggregates as the experiment bundle).

Method	nDCG@10	nDCG@100	RR@10	AP@100	P@10
BM25	0.3155	0.3773	0.2565	0.2690	0.0506
LTR α-blend (α=0.85)	0.3197	0.3793	0.2587	0.2711	0.0515
First-stage RRF3	0.3026	0.3629	0.2391	0.2507	0.0509
Learned fusion (OOF)	0.3859	0.4275	0.3200	0.3280	0.0596
CE cascade (MiniLM, top-32 on RRF3 prior)	0.4110	0.4514	0.3544	0.3610	0.0590

CE vs BM25 on nDCG@10: mean paired difference 0.09541; 95% bootstrap CI [0.08625, 0.10470] (5,193 topics, 4,000 resamples).

Corpus: MS MARCO document ranking v1 (3,213,835 documents). Training: MS MARCO passage triples.train.small. Data: Microsoft MS MARCO. This Hub repo does not redistribute that corpus.

Intended use

• Reproduction of the bi-encoder stage in the associated publications
• Pool-restricted hybrid reranking: BM25 (or another first stage) → this model → optional fusion / CE
• Query–passage scoring via LTRBertSIR (see Quick start)

Citation

@article{quadri2025poolrestricted,
  title   = {Empirical Analysis of Score Fusion Strategies under Pool-Restricted Dense Encoding for Ad Hoc Document Retrieval},
  author  = {Quadri, Syed Minnatullah and Chakkarwar, Vrishali A.},
  journal = {MSW Management Journal},
  url     = {https://mswmanagementj.com/index.php/home/article/view/4038}
}

@article{wang2024ltrbert,
  author  = {Wang, Junmei and Huang, Jimmy X. and Sheng, Jinhua},
  title   = {An Efficient Long-Text Semantic Retrieval Approach via Utilizing Presentation Learning on Short-Text},
  journal = {Complex \& Intelligent Systems},
  volume  = {10},
  pages   = {963--979},
  year    = {2024},
  doi     = {10.1007/s40747-023-01192-3}
}

Cite Bajaj et al. (MS MARCO) when using the dataset.

License

Apache-2.0 for the weights and code in this repository. MS MARCO data remain under Microsoft research terms and are not redistributed here.