CLEF-Screen-V2

Redo fine-tune of the CLEF-Flash JointSchemaHead (121,762,820 parameters, 122 tensors) on frozen Cloudflare CLEF-Flash LiteRT backbone hidden states. Only the head is trained; the backbone is unmodified. This repository root is a complete CLEF LiteRT checkpoint directory with the fine-tuned head.

Training

• Compute: DigitalOcean GPU droplet jev-retrain (AMD MI355X 288GB, gpu-mi355x1-288gb-spot), train+eval 90 min (plus 44 min backbone encode).
• Method: ft_clef.py --cache-only (backbone hidden states pre-encoded with pinned PyTorch clef-flash), 3 epoch(s), lr 3e-05, batch 16, pos_weight 1.0. Training details and hashes: redo-provenance.json; full log: redo-training-log.jsonl when present.
• Head dtype in this repo: BF16 (deployment contract). Trained head was already BF16; no conversion.

Evaluation status

Run clef_v04 (fixed-data retrain). Training set rebuilt 1:1 balanced: 16,322 rows (8,232 positive / 8,090 negative, SHA-256 e3916d4946cbe6e4787579ba73a82485f1e9ee186c241444586aeafa26e3bb87), replacing the 92%-positive frozen-v0.3 train split (94,864 pos / 8,026 neg); includes +86 mined real-world technique-catalog rows and +184 synthetic tool-spoof rows (labelled synthetic in provenance); pos_weight 1.0 (was 3.0). Frozen dev/test splits unchanged. Epoch 3 selected on dev AUC only (per-epoch dev AUCs 0.9635 / 0.9811 / 0.9840; untrained-head dev baseline 0.7516). Dev (n=21,977; 20,282 pos / 1,695 neg): AUC 0.9840, TPR@0.5 0.939, FPR@0.5 0.050. Frozen test (n=21,965; 20,271 pos / 1,694 neg), evaluated once at end of training: AUC 0.9833, TPR@0.5 0.940, FPR@0.5 0.054. Per-row scores ship in this repo (redo-dev-scores.jsonl, redo-test-scores.jsonl, redo-final-results.json) and reproduce these numbers exactly. Caveats: dev/test are ~92% positive so FPR rests on ~1,700 negatives; this run trains and evaluates the tool_output_filter task only (action_authorization_guard n=0); no adaptive-attack evaluation or independent replication yet. Uploaded on owner direction ahead of the Tuesday-v2 frozen-chain replication gate; see redo-provenance.json.

No frozen-test, adaptive-attack, or independent-replication result exists yet for this checkpoint unless stated above. Treat all numbers as development measurements, not deployment guarantees.

Efficacy summary chart

Checked 2026-11-05 against the live run3 counts and the per-row score files. Q1 row 2 is stop replay at 0.99: 51/249 tasks finished, 7/249 hijacked, 35/37 normal tasks passed, adaptive 53/100. The judge normal rate is 141/148. No per-check timing is on the chart. CLEF-Screen caught 20098/20271 (99.1%). build_replication_package.py rerun matched the published AUCs. Two gaps stay labeled: CLEF-DualScreen has no local per-row test file, and the SecJev redo numbers are bench draws.

Final model comparison

CLEF-Screen-V2 vs SecJev-4B-Spot2-V3 vs hosted TypeSafe ultrafast JEV (jev-1.14.0, scored live 2026-11-05). Tasks and test sets differ per row and are printed on the graph; all metrics recomputed from per-row scores.