SingleBicycle/aev-edl-heads

AEV — EDL Belief-Update Cascade Heads

Trained gating heads for the EDL belief-update cascade of the Agentic Evidential Verifier (AEV) project. Each incoming observation o_{t+1} passes through a gated cascade that maintains the agent's belief graph S_t = (V_t, E_t, M_t):

observation o_{t+1}
  └─ EDL_A admission {admit, ignore}                    K=2
       └─ admit → Atomizer + Proposer
            └─ EDL_M merge {merge, append}              K=2
                 └─ append → EDL_R relation {supports, contradicts, none}  K=3
                              none + high U → quarantine

Every head outputs Dirichlet evidence α = f_θ(·) + 1; uncertainty U = K / Σα. When U ≥ τ the argmax decision is overridden and the unit is quarantined. All DeBERTa heads are full fine-tunes of microsoft/deberta-v3-base (cross-encoder, max_len 256) trained with the Sensoy et al. 2018 EDL loss (Bayes-risk CE + annealed KL to uniform).

Checkpoints

Path	Head	Labels	test macro-F1	τ (dev)	Notes
edl_head_admission/	EDL_A	admit / ignore	0.965	0.136 → 94% cov @ 1.6% sel. risk	HotpotQA-distractor + FEVER+ mismatch mining + tau-bench noise
edl_head_admission_swe/	EDL_A (SWE-aligned)	admit / ignore	0.807 (silver)	budget15 fallback	1-epoch continued FT on 3k SWE-trace weak labels + 50% replay
edl_head_merge/	EDL_M-text	merge / append	0.888	0.230 → 82% cov @ 5.3% sel. risk	PAWS + QQP + VitaminC flagging revisions
edl_head_merge_swe/	EDL_M-text (SWE-aligned)	merge / append	0.959 (silver)	budget15 fallback	same recipe as admission_swe
edl_deberta/	EDL_R v0	supports / contradicts / none	0.877	0.150	wave-3 FEVER+ 763k checkpoint (FEVER + ANLI + VitaminC + Climate-FEVER), reused as relation head
dep_estimator.pt	DepNet (M-feat v0)	8-way dependency type	0.947 (type macro-F1)	—	774-d input MLP; posterior coarsened to merge/append for the M-feat arm

All DeBERTa heads: ECE ≤ 0.03, vacuity-AUROC 0.84–0.88 (in-domain).

Deployment caveat (validated twice): absolute τ does not transfer across domains — zero-shot on SWE traces EDL_A quarantines everything and EDL_M merges confidently-wrong. Use a conformal per-batch quantile instead: τ_eff = max(served τ, 85th percentile of batch U).

Format & loading

Each directory contains model.pt (state dict) and meta.json (labels, backbone, metrics, fitted τ). Load with the project trainer (repo agentic-evidential-verifier):

# PYTHONPATH=src, conda env aev
from aev.train_heads import load_head           # DeBERTa heads
head = load_head("edl_head_admission")           # or any directory above

Requires transformers>=5 gotchas: load with dtype=torch.float32 and the sentencepiece slow tokenizer (handled inside the project loader).

Results context

Full experimental report: EXPERIMENTS_REPORT_EDL_HEADS.md in the project repo (E0 dataset vetting → E1 head training → E2 domain alignment → E3 offline cascade benchmark: 100% noise interception, −93% residual redundancy → E4 live A/B on SWE-bench Lite: non-inferiority established).