epidebug-mm-v5

Option scorer over frozen all-MiniLM-L6-v2 embeddings, with hypothesis→evidence attention and tabular physics features. Its scorer uses interactions only, so it is invariant to option order by construction. Trained with randomized option order and a permutation-consistency KL, evidence-swap negatives, and the cause loss masked on AI4I mechanism rows. Selected run: seed 11, epoch 16, chosen on validation.

Bench v2.2 test (n=337, internal benchmark, not public)

model hyp follow-up cause
v3 .332 .365 .433
v4 .294 .288 .383
v5 .522 .487 .549
best surface baseline .214 .418 (mode length rank) —

Evidence probes (change in hyp acc): blank −.139, swap −.300. The gold_first probe inserts the gold cause into the evidence text, so it measures leak sensitivity, not option position: +.196.

Caveats: the follow-up longest-option baseline already scores .389, so follow-up's margin over surface heuristics is modest (+.07). Results are from a single benchmark and one selected seed. Load with ckpt/v4.py.

Downloads last month

-

Downloads are not tracked for this model. How to track
Safetensors
Model size
1.14M params
Tensor type
F32
·
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support