epidebug-mm-v6
This is v5, an interaction-only option scorer over frozen all-MiniLM-L6-v2 that is invariant to option order, with evidence-swap negatives and a permutation-consistency loss, plus:
- Option-only debiasing: a debiased product-of-experts with a train-time bias head that sees only the option texts (embedding plus row-relative style features: length rank, commas, digits, hedges). The main scorer is trained on
main + log p_bias.detach(), so it only has to explain what option style can't. A gradient-reversal adversary (Ξ»=0.3) predicts the gold from the scorer's option projections. Both heads are dropped at inference. - Temperature scaling fitted on validation (hyp 0.99, follow-up 0.94, cause 4.32). The temperatures are stored in the
Tbuffer and already applied to the outputs.
The selected run is seed 11, epoch 22, chosen by validation score. Load it with ckpt/v4.py.
Bench v2.4 test (n=337; internal benchmark, not public)
| model | hyp | follow-up | cause | ECE hyp / fu / cause |
|---|---|---|---|---|
| v5 | .401 | .472 | .549 | .121 / .151 / .325 |
| v6 | .478 | .665 | .585 | .091 / .037 / .139 |
| best surface baseline | .211 | .377 | β | |
| option-only style_gb (data probe) | .389 | .579 | β |
Memorization caveat: 75 of the 337 test hyp golds also appear as train golds. On those rows v6 scores .933 hyp / .987 fu (v5: .840 / .773). On the other 262 rows v6 scores .347 / .573 (v5: .275 / .385). Most of the headline hyp number comes from the overlap rows.
Probes (change in hyp acc): blank evidence β.240, evidence swap β.318, gold cause inserted into the evidence +.300. The permutation probe gives identical predictions under every ordering.
Style reliance (v6 accuracy when style_gb is right vs wrong): hyp .534 vs .442 (v5: .481 vs .350); follow-up .667 vs .662. The hyp gap is smaller than v5's but not gone.
Caption pairs (n=24, private): hyp .667 vs v5's .917, a regression on this small set. There were 0 flips among the 8 clean-correct pairs.