Nikunjgoyal07/jev-deberta-decider

JEV Decider — DeBERTa-v3-base + JEVHead (LoRA merged)

System-1 decision model: state + question in, typed probabilities out, zero generated tokens. Three question types: choice (argmax + full distribution), noul (P(yes)), score (expectation + per-level probs).

Architecture

• Backbone: microsoft/deberta-v3-base (184M) with the trained LoRA (r=8, α=16 on query/key/value, all 12 layers) merged in, fp32 (backbone/). Vocab 128,003 — base tokens plus [P]/[L] markers baked into the embedding matrix, so there is no resize step at load time.
• Head: JEVHead (head.pt, fp32) — task token + [L] label states attend (2 blocks), one scalar per label, softmax over options. Encoding [CLS] [P] prompt ([L] label)* [SEP] state [SEP]; order invariance is trained (shuffle augmentation), not structural.
• Calibration: temperature T=1.25 fit on val.
• Runtime deps are just torch, transformers, sentencepiece — no peft, no torchao issues.

Training — honest status: single partial epoch

• Data: Nikunjgoyal07/jev-diverse-v1 (63,480 Q). Ran 1 epoch = 221 optimizer steps (batch 4×8), dual-LR AdamW, on A100.
• Measured test metrics (training log): acc 0.462 (choice 0.267 / score 0.228 / noul 0.682), auc_noul 0.755, ECE 0.027. This checkpoint is undertrained — choice/score are near chance. Pushed as-is for continuity; retrain with more epochs before relying on it.
• The merged backbone is bit-identical in behavior to the training artifact (verified: same outputs to 1e-7).

Use

from transformers import AutoModel, AutoTokenizer
REPO = "Nikunjgoyal07/jev-deberta-decider"
tok = AutoTokenizer.from_pretrained(REPO)
if tok.pad_token is None: tok.pad_token = tok.sep_token
net = AutoModel.from_pretrained(REPO, trust_remote_code=True).eval()
out = net.answer(tok, "Restore my account access, please.",
    {"intent": {"type": "choice", "instructions": "What does the customer want?",
                "criteria": {"account": "access problems", "billing": "payment issues"}},
     "urgent": {"type": "noul", "instructions": "Is there a deadline?"}})

trust_remote_code=True loads modeling_jev_deb.py from this repo (model classes verbatim from the training notebook) — nothing to install by hand. If downloads complain about rate limits, run huggingface-cli login first (needs a free HF token). Data licenses: several training slices are marked verify-before-commercial-use — see the dataset card.