SargeDev/jev-gate-student-b

Jev-Gate Student B — LoRA Memory-Relevance Judge

LoRA adapter (r=16, α=32, q_proj/v_proj) on Qwen/Qwen2.5-0.5B-Instruct, distilled from the Jev typed-judgment API into a compact local judge for agent-memory gating. Given a query + candidate memory passage, outputs P(relevant) via the calibrated yes probability over final-token logits.

Trained on SargeDev/jev-distill-corpus — paired relevance judgments (graded 0–7 + binary) distilled from typed judgment calls by a larger teacher.

Eval (held-out, n=60): MAE 0.187 / Pearson 0.791 / agreement 90.0% vs vanilla Qwen2.5-0.5B 0.536 / -0.067 / 38.3%. ~59 ms/judgment on RTX 3060, zero API cost.

Usage

import torch
import torch.nn.functional as F
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

BASE = "Qwen/Qwen2.5-0.5B-Instruct"
tok = AutoTokenizer.from_pretrained("SargeDev/jev-gate-student-b")
model = PeftModel.from_pretrained(
    AutoModelForCausalLM.from_pretrained(BASE, dtype=torch.bfloat16),
    "SargeDev/jev-gate-student-b",
).eval()

yes = tok(" yes", add_special_tokens=False).input_ids[0]
no = tok(" no", return_tensors=None, add_special_tokens=False).input_ids[0]

def p_relevant(query: str, text: str) -> float:
    prompt = (f"Memory: {text[:600]}\nQuery: {query}\nQuestion: Is this memory "
              "relevant for answering the query? Answer yes or no with confidence.")
    ids = tok(prompt, return_tensors="pt", truncation=True, max_length=384)
    with torch.no_grad():
        logits = model(**ids).logits[0, -1, :]
    return float(F.softmax(torch.tensor([float(logits[no]), float(logits[yes])]), dim=0)[1])

Gate at threshold 0.5. Fail-open on errors.