Eklav-Reranker-MaskOnly-Qwen3-8B

LoRA adapter for passage reranking, trained with the eklav-mask-only ablation method on base model Qwen/Qwen3-8B, from the Eklav project.

Note: intermediate checkpoint. This is step 4500 of 5968 (~75% through training), uploaded before training finished so an early version is available. It will be superseded by the final checkpoint once training completes.

Method

eklav-mask-only is an ablation, not a standalone method: it isolates Eklav's loss-masking trick as the single changed factor relative to std-SFT, holding everything else fixed. Concretely:

• Data: identical to std-SFT's data — full teacher reasoning trace in the response (<think>...long CoT...</think> true|false), no prompt-side reasoning hint, no lexical filtering of verdict-bearing sentences.
• Masking: the <think>...</think> span is present in the training target (the model sees it in context) but is masked out of the loss — only the post-</think> answer tokens are supervised.

This isolates masking (one of Eklav's three simultaneous changes vs. std-SFT) from the other two: prompt-side context placement and lexical filtering.

This is not the same as answer-only SFT, which never includes a <think> block in its training data at all. See AdarshSingh7647/Eklav-Reranker-AnswerOnly-Qwen3-4B for that method.

Training data

Same underlying data as std-SFT: AdarshSingh7647/Eklav-Reranker-CotGen-Data

Usage

LoRA adapter (PEFT format, r=32, alpha=64). Load with the base model:

from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

base = AutoModelForCausalLM.from_pretrained("Qwen/Qwen3-8B")
model = PeftModel.from_pretrained(base, "AdarshSingh7647/Eklav-Reranker-MaskOnly-Qwen3-8B")
tokenizer = AutoTokenizer.from_pretrained("AdarshSingh7647/Eklav-Reranker-MaskOnly-Qwen3-8B")

Part of the Eklav project.