TabRankSingleTableNaive

A single call generative listwise reranker for table retrieval built on Qwen3 8B. Given a question and a set of candidate tables the model returns the tables ordered from most to least useful for answering the question. It reads all candidates in one prompt and emits the full ranking in a single generation.

This checkpoint is the Single Table Answer Only variant.

What this variant does

The model is trained to output the ranking directly. Reasoning traces were shown during training but masked from the loss so the model learns to rank without writing a chain of thought. This makes it the fastest variant at inference.

Training data

This model is trained on the NQ Tables training split only which contains single table retrieval questions. The sibling checkpoint TabRankMultiTableNaive uses the other training mix with the same objective.

Input and output format

The user message lists candidate tables as blocks headed by ### Table 1 ### Table 2 and so on. The model returns a JSON object whose value is the ranked list of one based candidate positions best first:

{"ranked_tables": [3, 1, 5, 2, 4]}

Map those positions back to your table ids to obtain the reranked list.

Evaluation

Scored as a listwise reranker that reorders a first stage top 25 candidate list on four table question answering benchmarks. SQA and TAT QA use the full test split. HybridQA and TabFact use a fixed shared 500 query sample. acc@10 counts a query correct only when every gold table falls inside the top 10.

Dataset	n	recall@10	ndcg@10	acc@10	MRR
SQA	148	0.838	0.715	0.743	0.706
TAT QA	362	0.651	0.507	0.428	0.568
HybridQA	500	0.825	0.727	0.698	0.780
TabFact	500	0.743	0.673	0.524	0.777
Mean		0.764	0.655	0.598	0.708

Usage with vLLM

from vllm import LLM, SamplingParams
from transformers import AutoTokenizer

repo = "AdarshSingh7647/TabRankSingleTableNaive"
tok = AutoTokenizer.from_pretrained(repo)
llm = LLM(model=repo, dtype="bfloat16", max_model_len=32768)

system = ("You are a table relevance expert. Given a question and a set of candidate tables "
          "rank them from most to least useful for answering the question. Reason in a "
          "<think>...</think> block then output exactly JSON with key ranked_tables.")
user = "Question: ...\n\n### Table 1\n...\n\n### Table 2\n...\n"

msgs = [{"role": "system", "content": system}, {"role": "user", "content": user}]
text = tok.apply_chat_template(msgs, tokenize=False, add_generation_prompt=True)
# the model outputs the ranking json directly
out = llm.generate([text], SamplingParams(temperature=0.6, top_p=0.95, max_tokens=8192))
print(out[0].outputs[0].text)

Usage with Transformers

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

repo = "AdarshSingh7647/TabRankSingleTableNaive"
tok = AutoTokenizer.from_pretrained(repo)
model = AutoModelForCausalLM.from_pretrained(repo, torch_dtype=torch.bfloat16, device_map="auto")

system = ("You are a table relevance expert. Given a question and a set of candidate tables "
          "rank them from most to least useful for answering the question. Reason in a "
          "<think>...</think> block then output exactly JSON with key ranked_tables.")
user = "Question: ...\n\n### Table 1\n...\n\n### Table 2\n...\n"

msgs = [{"role": "system", "content": system}, {"role": "user", "content": user}]
text = tok.apply_chat_template(msgs, tokenize=False, add_generation_prompt=True)
inputs = tok(text, return_tensors="pt").to(model.device)
out = model.generate(**inputs, max_new_tokens=8192, temperature=0.6, top_p=0.95, do_sample=True)
print(tok.decode(out[0][inputs.input_ids.shape[1]:], skip_special_tokens=True))

Model details

• Base model Qwen3 8B
• Method LoRA rank 16 fine tuning merged into the base weights so it loads directly
• Precision bfloat16 single file safetensors near 16 GB
• Family the six TabRank checkpoints span three objectives (Answer Only, Reasoning Generation, Reasoning Conditioned) across two training mixes (Single Table, Single plus Multi Table)

Citation

The MultiTabQA data comes from RAG over Tables. Please cite it when using the Single plus Multi Table checkpoints:

@misc{zou2025ragtableshierarchicalmemory,
      title={RAG over Tables: Hierarchical Memory Index, Multi-Stage Retrieval, and Benchmarking},
      author={Jiaru Zou and Dongqi Fu and Sirui Chen and Xinrui He and Zihao Li and Yada Zhu and Jiawei Han and Jingrui He},
      year={2025},
      eprint={2504.01346},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={https://arxiv.org/abs/2504.01346}
}

The accompanying TabRank paper is currently under review. A citation will be added here once it is available on arXiv.