Universal Activation Oracle v20 — zero-shot
ONE Qwen3-1.7B+LoRA trunk that reads the mean-pooled activation of ANY LLM (via per-model linear
enc→2048, marker injection) and answers calibrated Yes/No questions about latent behaviours. Trained
on a broad union of 28 concepts (reward-model quirks + social/political biases + CoT-correctness),
it detects concepts it never trained on, zero-shot.
Headline (held-out model llama3-8b)
| v19 (narrow, 6 concepts) | v20 (broad, 17 concepts) | |
|---|---|---|
| supervised mean AUROC | 0.950 | 0.988 |
| clean-FP (confabulation) | 0.117 | 0.018 |
| zero-shot on held-out concepts | ~0.58 (always-No) | ~0.97 |
Zero-shot AUROC on concepts NEVER trained: decimal 1.00, atomic 1.00, chinese_bias 1.00, muslim_bias 1.00, movie 0.99, chocolate 1.00.
The lever is training breadth, not architecture. A narrow detector answers "No" to anything off-vocabulary; a broad one learns the general skill of activation-property detection and transfers to unseen concepts — matching a general multi-layer AO (niclas-luick MLAO) reproduced at 0.86 zero-shot on these biases, here in our own harness with the transfer-friendly embedding-marker injection.
Contents
v20_broad/: trunk LoRA (av/), enc/dec bundle (adapters/),v18_meta.json.eval_v20.json(zero-shot + supervised), plus the v19 social-bias evals and the MLAO head-to-head.RESULTS.md: full write-up (v19 social biases → cot fold-in → MLAO reproduction → injection ablation → v20 breadth→zero-shot).
Code: github.com/AlexWortega/qwen3-1p7b-nla — scripts/audit/{train_v18,eval_v18,merge_xmodel,run_mlao_ref}.py.