Instructions to use FDU-INS/ClosureRAG with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use FDU-INS/ClosureRAG with Transformers:
# pip install -U transformers accelerate # Load model directly from transformers import AutoModel model = AutoModel.from_pretrained("FDU-INS/ClosureRAG", device_map="auto") - Notebooks
- Google Colab
- Kaggle
ClosureRAG cross-encoders
This repository holds the trained checkpoints of ClosureRAG, a retriever for questions decided by a set of interdependent norms. A retrieval-augmented system answers such a question correctly only if it retrieves the whole set, the evidence closure.
- Code, benchmark data (InsClosure) and the results behind the paper's tables: https://github.com/vivilinx66-maker/ClosureRAG
All three models are cross-encoders fine-tuned from Qwen3-Reranker-0.6B. Each scores a (case, passage) pair by the yes/no logit difference at the last position.
| Folder | Role in ClosureRAG | Test (1,160 questions): FullCov@5/10/20 / R@5/10/20 |
|---|---|---|
l1_coverage_ranker/ |
Level 1: group-wise coverage ranker. Scores every candidate of the 400-passage pool on its own; its scores are fused with the first-stage rank (μ = 1.0). | ClosureRAG-L1: 20.8 / 38.3 / 75.1 / 60.4 / 76.2 / 91.1 |
l2_context_rescorer/ |
Level 2: context-conditioned rescorer. Rescores the fused top-60 given the other top-ranked passages (m = 8) and is mixed with the fused score (λ = 0.5). | ClosureRAG: 27.6 / 52.2 / 81.3 / 69.1 / 82.0 / 93.5 |
l1_pointwise_ablation/ |
Ablation: the level-1 ranker trained with a pointwise loss instead of the group-wise loss (fused with μ = 1.5). | 15.7 / 31.4 / 66.2 / 61.3 / 72.2 / 87.6 |
Each folder holds a complete transformers checkpoint (model.safetensors, tokenizer, configs) and train_log.json with the training
hyper-parameters.
Usage
The models expect the input format of the ClosureRAG code: the case text, a candidate passage with its document card and provenance, and, for level 2, the context cards and graph relations. Use them through the GitHub repository:
git clone https://github.com/vivilinx66-maker/ClosureRAG && cd ClosureRAG
pip install -r requirements.txt
hf download FDU-INS/ClosureRAG --local-dir models # the three folders, about 3.6 GB
bash scripts/run_inference.sh test models/l1_coverage_ranker models/l2_context_rescorer
run_inference.sh runs the whole online pipeline on the released candidate pools:
- Build the cross-encoder requests (
make_requests.py). - Level-1 scoring (
score_l1.py). - Rank fusion with the first stage (
fuse.py). - Level-2 rescoring (
score_l2.py). - Final ranking (
final_rank.py) and evaluation.
It takes about one hour for the 1,160 test questions on an A100-40GB. Download a single model with --include "l1_coverage_ranker/*".
Scoring individual candidates in Python (run from the repository root, so that closurerag and data/ are found):
import torch
from transformers import AutoTokenizer, AutoModelForCausalLM
from closurerag.fmt import Fmt, score
path = "models/l1_coverage_ranker"
tok = AutoTokenizer.from_pretrained(path, padding_side="left")
model = AutoModelForCausalLM.from_pretrained(path, dtype=torch.bfloat16).cuda().eval()
F = Fmt(tok, 768) # reads the passages and document cards from data/
case = F.case(question_text) # the question text of a benchmark item
seqs = [F.encode(case, F.cand(cid, why)) for cid, why in candidates] # candidate passage ids + provenance text
with torch.no_grad():
s = score(model, tok, F, seqs) # higher = more needed to decide the case
Level 2 uses the same Fmt with closurerag.fmt.encode_set and a context of passage ids. See scripts/score_l2.py.
Training
Training data, lists and scripts are in the GitHub repository (scripts/run_training.sh). All models were trained on one A100-40GB in bf16
with AdamW (learning rate 1e-5) and 8 lists per update.
- Level 1: 2,058 lists, 2 epochs, 516 updates; group-wise loss with τ = 1, up to two members per gold group, 24 hard and 8 random negatives, pointwise term of weight 0.2.
- Level 2: 6,174 lists (three contexts per question), 1 epoch, 772 updates; 12 hard and 6 random negatives, pointwise term of weight 0.05.
Licence
Apache-2.0, as the base model Qwen3-Reranker-0.6B.