hindsight-v2

A small, calibrated, non-generative verifier for coding-agent sessions. Given the canonical event trace of a session (user turns, agent messages, tool calls and results, diffs) it predicts, at every user-turn boundary, whether the user's next turn will be a correction, and of which kind: not-done, unrequested, rule-violation, other. Trained on hindsight labels (what the user actually did next), never on the agent's own claims. Encoder: UniXCoder-base fine-tuned end to end; level 2: a 2-layer GRU over the last 24 event vectors; heads Platt-calibrated on a held-out split.

Held-out AUROC (Platt-scaled, test split by repository and user)

head	all sources	Claude Code local	Codex local	SWE-chat
correction	0.727	0.743	0.709	0.719
not_done	0.778	0.841	0.747	0.767
unrequested	0.845	0.926	0.744	0.857
rule_violation	0.678	0.772	0.865	0.661
other	0.599	0.627	0.714	0.576

Coverage at 80% precision on the any-correction head is 0.04; use it as a rank or reject signal. The other head is not usable.

Files

encoder/ (transformers, safetensors), model.pt (level-2 core and heads, keys prefixed ver.), config.json (dims, window, source list), results.json (metrics, Platt parameters), quant_bench.json (fp32 vs int8 vs ONNX on CPU: keep fp32).