Learned activation priors for steering GPT-2 small
Model weights for the study Cheap learned priors for activation steering.
Code, protocols and the full report: https://github.com/zxclwnq/steering-denoisers
Headline result
A learned activation prior does not provide useful orthogonal naturalization of a steering intervention. Each model below learns its own training task well, and none of them beats a plain hard clamp on language-model quality once the semantic coordinate is held fixed.
The decisive comparison is the constraint-preserving tangent flow:
| criterion | value | outcome |
|---|---|---|
| T1, recovered fraction of the damage (threshold 0.25) | 0.7730 | criterion met |
| T2, pooled ΔNLL against a hard clamp | +0.006184 nats, 95% CI [+0.001631, +0.010788] | worse than the clamp |
A hard clamp costs only +0.003 to +0.054 nats while satisfying the coordinate exactly.
Contents
Every directory holds model.pt (a training checkpoint containing model state dict
plus optimizer/step metadata), best.json (the selected step and its validation loss)
and training_status.json (the completed-run record).
| directory | what it is | params | SHA256 of model.pt |
|---|---|---|---|
tangent-flow-16m-linear/ |
constraint-preserving tangent flow, linear path. The headline model. | 16M | 066afb60…4199d4c |
tangent-flow-16m-variance-preserving/ |
same objective on a variance-preserving path | 16M | 1047d063…d87a25fcc |
steering-corruption-denoiser-16m/ |
denoiser trained on steering-like corruption z = h + δv |
16M | 8b95467b…5e566c3c2 |
flow-prior-16m/ |
unconditional flow-matching activation prior | 16M | 70f8999d…4d64b298 |
flow-prior-60m/ |
the same prior at larger capacity | 60M | 68482e68…c36affb1 |
conditional-flow-60m/ |
direction-coordinate conditional flow | 60M | 83324cfa…41933f76 |
tangent-flow-16m-linear/model.pt is the same checkpoint published separately as
qweclownq/tangent-flow-16m,
which additionally carries the T1/T2 evaluation receipts.
Where the activations come from
All models operate on the GPT-2 small residual stream at layer 7 (blocks.7.hook_resid_post),
read through TransformerLens. from_pretrained applies fold_ln and center_writing_weights,
so this basis is numerically not the input to transformer.h[7] in plain HuggingFace
transformers. Reproducing the numbers outside TransformerLens requires reproducing that
transform first.
Inputs are standardized as x = (h - mu) / sigma with statistics from the training
activation set. Flow models predict the velocity u = eps - x_0 along
x_t = (1 - t) x_0 + t eps.
Loading
import torch
ckpt = torch.load("tangent-flow-16m-linear/model.pt", map_location="cpu")
state = ckpt["model"]
The matching architecture definition and inference code are in the GitHub repository;
these files are raw training checkpoints, not transformers-compatible models.
Training data
Activations collected from FineWeb text. Steering directions come from
jbloom/GPT2-Small-SAEs-Reformatted. Held-out directions were never accessed:
every artifact of the project records held_out_accessed: false.
Citation
See the repository for the report and the frozen protocol documents behind each number.