jprivera/260924_full_eval_charts

Full-eval charts (panel + remainder), 2026-09-24

Built from the full held-out monitor eval (7,200 panel + 10,584 remainder rows per arm, all 96 arms), graded by a copy of the grader in this folder. Nothing in 260923_paired_grid/ or 260923_paper_figures/ was changed.

Start here: SUMMARY_FOR_PAPER_260924.md: bottom line, claims, protocol facts, known issues, policy status.

Also: EVAL_SIZE_ANALYSIS.md: the monitor eval is 5–7× larger than the headline needs.

Figures

• Fig 1: overview, uncaught collusion remaining after 3 passes
• Fig 2: collusion remaining per behaviour
• Fig 3: dose curves, 0/1/2/3 passes
• Fig 4: collateral costs

Bar charts (the overview set, full data)

• Remaining collusion, original vs 3 passes
• Transfer
• Dose curves
• By behaviour, v2 (reds), v2 (red/green)
• Costs
• Capability, GSM8K + MMLU (capability data is unchanged by the remainder)

Data and scripts

• report/grid.json: graded results for all 96 arms (status COMPLETE)
• report/tables.md, report/dose_tables.md: tables
• fig1_numbers.json: numbers behind Fig 1
• scripts/06_grade_full.py: grader copy. It accepts the panel-half manifest sha that the merge left in each arm's ADAPTER_SHA.json and still enforces the exact row-id set against the full manifest.
• scripts/make_figures_full.py, scripts/plot_*_full.py: figure scripts (copies pointed at the full grid; outputs only in this folder)

Notes

• UltraChat (control) is panel only (n=300 per cell); its remainder was never run. It is labelled that way in the figures.
• Compared with the panel-only figures, every method moves by 2 points or less.