Yuanbao-Reranker-8B

Document-Grounded Synthesis and Preference Distillation for Text Reranking

0.6B · 4B · 8B · GitHub Repository

Introduction

Yuanbao-Reranker-8B is the 8B member of the Yuanbao-Reranker family (0.6B, 4B, and 8B) built upon the Qwen3 backbone. Given a task instruction, a search query, and candidate documents, it outputs fine-grained relevance scores in [0, 1] across Chinese, multilingual, English, long-document (up to 8,192 tokens), and code retrieval scenarios.

• Cross-Tier Ranking Performance: Across five comprehensive benchmark panels (CMTEB-R, MMTEB-R, BEIR, MLDR, and MTEB-Code), Yuanbao-Reranker-8B reaches a state-of-the-art 74.21 Avg., outperforming Qwen3-Reranker-8B (72.68) by 1.53 points and achieving the highest score on every individual panel.
• Document-Grounded Synthesis & Stratified Group Curation: Trained on 2.75M 1:3 ranking groups constructed via answer-first and query-type-guided synthesis, a three-stage curation funnel (teacher verification, semantic deduplication, and source-by-length quota allocation), and teacher-screened (Δ > 1) difficulty-stratified negative mining (Hard, Medium, and Moderate tiers).
• Bounded Minimum-Gap Preference Distillation: Jointly optimizes pointwise yes/no cross-entropy and a capped one-sided pairwise hinge objective over positive–negative and negative–negative pairs, followed by layerwise Adaptive LERP model merging with zero inference overhead.

Model Family & Specifications

Model	Parameters	Layers	Context Length	Weight Format	Five-Panel Avg.
Yuanbao-Search/Yuanbao-Reranker-0.6B	0.6B	28	8,192	Safetensors, BF16	70.96
Yuanbao-Search/Yuanbao-Reranker-4B	4B	36	8,192	Safetensors, BF16	73.63
Yuanbao-Search/Yuanbao-Reranker-8B	8B	36	8,192	Safetensors, BF16	74.21

• Input: Task instruction (<Instruct>), search query (<Query>), candidate document (<Document>)
• Output: Relevance probability in [0, 1] normalized over the yes and no decision tokens
• Context Length: Up to 8,192 tokens

Quickstart: Scoring Query–Document Relevance

You can compute query–document relevance scores directly with standalone Hugging Face Transformers or vLLM (no external wrapper required), or via our yuanbao-reranker package.

1. Standalone Usage with Hugging Face Transformers

import math
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

SYSTEM = (
    'Judge whether the Document meets the requirements based on the Query and the Instruct provided. '
    'Note that the answer can only be "yes" or "no".'
)
DEFAULT_INSTRUCTION = "Given a web search query, retrieve relevant passages that answer the query"
PREFIX = f"<|im_start|>system\n{SYSTEM}<|im_end|>\n<|im_start|>user\n"
SUFFIX = "<|im_end|>\n<|im_start|>assistant\n<think>\n\n</think>\n\n"

model_name = "Yuanbao-Search/Yuanbao-Reranker-8B"
tokenizer = AutoTokenizer.from_pretrained(model_name, trust_remote_code=False)
if tokenizer.pad_token_id is None:
    tokenizer.pad_token = tokenizer.eos_token

model = AutoModelForCausalLM.from_pretrained(
    model_name, torch_dtype=torch.bfloat16, device_map="auto", trust_remote_code=False
).eval()

yes_id = tokenizer.encode("yes", add_special_tokens=False)[0]
no_id = tokenizer.encode("no", add_special_tokens=False)[0]
suffix_ids = tokenizer.encode(SUFFIX, add_special_tokens=False)

def encode_pair(query: str, document: str, instruction: str = DEFAULT_INSTRUCTION, max_length: int = 8192):
    content = f"<Instruct>: {instruction}\n\n<Query>: {query}\n\n<Document>: {document}"
    body = tokenizer.encode(PREFIX + content, add_special_tokens=False)[:max_length]
    return body + suffix_ids

@torch.inference_mode()
def score_pairs(pairs, instruction: str = DEFAULT_INSTRUCTION, max_length: int = 8192):
    encoded = [encode_pair(q, d, instruction, max_length) for q, d in pairs]
    batch = tokenizer.pad({"input_ids": encoded}, padding=True, return_attention_mask=True, return_tensors="pt")
    batch = {k: v.to(model.device) for k, v in batch.items()}
    mask = batch["attention_mask"]
    positions = (mask.long().cumsum(-1) - 1).masked_fill(mask == 0, 0)
    logits = model(**batch, position_ids=positions, use_cache=False).logits
    last = torch.arange(mask.shape[1], device=mask.device).expand_as(mask).masked_fill(mask == 0, -1).max(dim=1).values
    selected = logits[torch.arange(len(encoded), device=mask.device), last][:, [yes_id, no_id]].float()
    logprobs = selected.log_softmax(dim=-1).cpu().tolist()
    return [math.exp(y) / (math.exp(y) + math.exp(n)) for y, n in logprobs]

query = "What is the capital of China?"
documents = [
    "Beijing is the capital of China.",
    "Shanghai is a major financial center.",
    "北京是中国的首都。",
]
scores = score_pairs([(query, doc) for doc in documents])
for doc, score in sorted(zip(documents, scores), key=lambda x: x[1], reverse=True):
    print(f"Score: {score:.4f} | Document: {doc}")

2. Standalone Usage with vLLM

import math
from transformers import AutoTokenizer
from vllm import LLM, SamplingParams
from vllm.inputs import TokensPrompt

SYSTEM = (
    'Judge whether the Document meets the requirements based on the Query and the Instruct provided. '
    'Note that the answer can only be "yes" or "no".'
)
DEFAULT_INSTRUCTION = "Given a web search query, retrieve relevant passages that answer the query"
PREFIX = f"<|im_start|>system\n{SYSTEM}<|im_end|>\n<|im_start|>user\n"
SUFFIX = "<|im_end|>\n<|im_start|>assistant\n<think>\n\n</think>\n\n"

model_name = "Yuanbao-Search/Yuanbao-Reranker-8B"
tokenizer = AutoTokenizer.from_pretrained(model_name, trust_remote_code=False)
yes_id = tokenizer.encode("yes", add_special_tokens=False)[0]
no_id = tokenizer.encode("no", add_special_tokens=False)[0]
suffix_ids = tokenizer.encode(SUFFIX, add_special_tokens=False)

llm = LLM(
    model=model_name, dtype="float16", max_model_len=10000,
    enable_prefix_caching=True, gpu_memory_utilization=0.8, trust_remote_code=False,
)
sampling = SamplingParams(
    temperature=0, top_p=0.95, max_tokens=1, logprobs=20, allowed_token_ids=[yes_id, no_id],
)

def score_pairs_vllm(pairs, instruction: str = DEFAULT_INSTRUCTION, max_length: int = 8192):
    prompts = []
    for q, d in pairs:
        content = f"<Instruct>: {instruction}\n\n<Query>: {q}\n\n<Document>: {d}"
        body = tokenizer.encode(PREFIX + content, add_special_tokens=False)[:max_length]
        prompts.append(TokensPrompt(prompt_token_ids=body + suffix_ids))
    outputs = llm.generate(prompts, sampling, use_tqdm=False)
    scores = []
    for out in outputs:
        lp = out.outputs[0].logprobs[-1]
        yes = -10.0 if yes_id not in lp else float(lp[yes_id].logprob)
        no = -10.0 if no_id not in lp else float(lp[no_id].logprob)
        y_p, n_p = math.exp(yes), math.exp(no)
        scores.append(y_p / (y_p + n_p))
    return scores

3. Using the yuanbao-reranker Package

Clone our GitHub repository to use the unified Python API or CLI with either backend="vllm" or backend="transformers":

from yuanbao_reranker import YuanbaoReranker

model = YuanbaoReranker("Yuanbao-Search/Yuanbao-Reranker-8B", backend="transformers")
results = model.rerank(
    query="Sort a list of integers in ascending order.",
    documents=["sorted(values)", "sum(values)"],
    instruction="Given a code comment, retrieve the code snippet corresponding to that comment.",
    top_n=2,
)
print(results)

Recommended Task Instructions

Yuanbao-Reranker is instruction-aware. You can customize <Instruct> to specify the target relevance criteria for your scenario:

Retrieval Scenario	Recommended instruction
General Web Search & RAG (Default)	Given a web search query, retrieve relevant passages that answer the query
Chinese Web Passage Retrieval	Given a Chinese search query, retrieve web passages that answer the question
Long-Document Retrieval (up to 8K)	Given a web search query, retrieve relevant passages that answer the query
Natural Language to Code Search	Given a code comment, retrieve the code snippet corresponding to that comment.
Technical & Coding QA	Given a question about coding, retrieval code or passage that can solve user's question
Fact & Claim Verification	Given a claim, retrieve documents that support or refute the claim
Legal Statute & Case Retrieval	Identifying the most relevant statutes for a given situation

Tip: For custom domain tasks, formulate a concise English instruction describing what constitutes a relevant document for the query (e.g., "Given a financial inquiry, retrieve relevant earnings report passages that answer the question").

Benchmark Results

1. Main Results on Qwen Top-100

All models rerank the shared Qwen Top-100 candidate pool (Top-100 candidates per query retrieved by Qwen3-Embedding-0.6B) across five comprehensive benchmark panels (nDCG@10 × 100):

Model	Params (Act. / Tot.)	CMTEB-R	MMTEB-R	BEIR	MLDR	MTEB-Code	Avg.
Sub-1B Compact Models (< 1B Active)
mxbai-rerank-base-v2	0.5B	69.77	59.04	58.22	42.99	68.01	59.61
bge-reranker-v2-m3	0.57B	72.14	58.27	55.48	59.41	41.34	57.33
Querit-Reranker-A0.4B	0.43B / 4.92B	70.44	61.42	57.78	56.55	65.78	62.39
KaLM-Reranker-Nano-R2	0.27B / 0.79B	70.00	62.26	58.47	66.13	65.38	64.45
jina-reranker-v3.5	0.6B	73.96	66.71	61.32	65.01	73.90	68.18
Qwen3-Reranker-0.6B	0.6B	71.43	65.61	59.61	66.89	73.03	67.32
Yuanbao-Reranker-0.6B	0.6B	75.60	70.49	61.30	68.94	78.44	70.96
Mid-Sized Models (1B–4B Active)
KaLM-Reranker-Small-R2	1B / 2.12B	73.18	64.17	60.90	67.29	69.09	66.93
mxbai-rerank-large-v2	1.54B	72.09	64.74	60.23	54.25	72.78	64.82
bge-reranker-v2-gemma	2.51B	72.35	65.89	60.29	47.30	64.26	62.02
Querit-Reranker-Dense-4B	4B	73.02	66.55	59.87	49.98	73.76	64.64
KaLM-Reranker-Large-R2	4B / 7.51B	75.58	66.86	62.72	69.72	73.75	69.73
Qwen3-Reranker-4B	4B	76.05	71.29	63.66	69.35	80.33	72.13
Yuanbao-Reranker-4B	4B	78.31	74.08	64.74	70.04	80.98	73.63
Large Models (8B)
Qwen3-Reranker-8B	8B	77.61	70.90	64.74	69.63	80.50	72.68
Yuanbao-Reranker-8B	8B	78.99	74.84	65.51	70.12	81.58	74.21

2. Cross-Retriever Generalization on Harrier Top-100

Evaluations on Top-100 candidate pools retrieved by Harrier-OSS-v1-0.6B verify that these gains generalize robustly across distinct first-stage candidate generators:

Model	CMTEB-R	MMTEB-R	BEIR	MLDR	MTEB-Code	Avg.
Qwen3-Reranker-0.6B	71.16	67.55	60.30	63.25	72.90	67.03
Yuanbao-Reranker-0.6B	75.41	74.81	61.68	65.29	78.15	71.07 (+4.03)
Qwen3-Reranker-4B	75.74	73.18	64.24	66.01	79.92	71.82
Yuanbao-Reranker-4B	78.11	81.82	65.33	65.98	80.60	74.37 (+2.55)
Qwen3-Reranker-8B	77.39	72.87	65.37	65.74	80.13	72.30
Yuanbao-Reranker-8B	78.76	81.21	66.02	66.24	81.15	74.68 (+2.37)

Training and Evaluation Pipeline

The GitHub repository provides the complete training, inference, and benchmark evaluation codebase.

Citation

@article{yuanbao_reranker_2026,
  title={Yuanbao-Reranker: Document-Grounded Synthesis and Preference Distillation for Text Reranking},
  author={Xu, Hongyuan and Ye, Dezhi and Cao, Jiahao and Liu, Jie and Liang, Haijin and Shi, Haibo and Ma, Jin},
  journal={arXiv preprint},
  year={2026}
}

Acknowledgments

This model is built upon the Qwen3 model family.