reinworld99/sae-null-step

Null-step control for SAE feature drift

Artifact for Retraining Is the Confound: A Null-Step Calibration for Sparse-Autoencoder Feature Drift Across Fine-Tuning Checkpoints. Anonymous for double-blind review.

An SAE fitted at every fine-tuning checkpoint, each warm-started from the last, is a standard way to watch features move. Between two adjacent checkpoint SAEs the model changed, but the SAE was also retrained, and retraining alone moves the basis. The null step isolates that: retrain the base SAE under the identical recipe on activations from the frozen base model, so any drift it produces is retraining.

Layout

tables/            every number in the paper, as CSV, plus the code that makes them
run_metadata/      per-run hyperparameters and training logs
weights/
  chains/          112 checkpoint SAEs, 5 fine-tuning chains x 4 layers
  null_step/       the control: 1 canonical + 12 seeds per layer
  null_trajectories/  7-step null chains, 4 layers x 3 seeds
  controlled_rerun/   base SAEs for the one-split TopK / JumpReLU rerun
  real_arm_L18/       per-step decoders, both arms, 4 seeds each
  tuned_lens/         per-layer tuned lenses for the token-overlap metric

What backs what

Paper element	Path
§3.1 fidelity, 118 evaluations	tables/sae_checkpoint_metrics.csv
§3.1 inherited-index alignment	tables/drift_out/drift_by_stage.csv
Table 1, controlled rerun	tables/controlled_rerun/, tables/real_arm/
Figure 1c, null trajectory	tables/drift_out/null_trajectory_l18_seeds.csv
Figure 1d and §3.4, direction test	tables/real_arm/L18_pairwise_cosines.csv
§3.2, artifact null step	tables/drift_out/null_replicates.csv
§3.3, checkpoint selection	tables/drift_out/checkpoint_selection.csv
§3.3, shuffled reward	weights/chains/sae_shuffled/, tables/task_performance.csv
§3.5, architecture arms	tables/controlled_rerun/jumprelu_*.csv
Appendix H, heterogeneity	tables/drift_out/drift_per_feature.csv

Reproducing the numbers

export SAE_RL_ROOT=/path/to/this/repo
python3 tables/scripts/paper_numbers.py --json > fresh.json
diff fresh.json tables/expected/paper_numbers_full.json    # expect no output

tables/scripts/make_numbers.py emits the LaTeX macros the manuscript quotes, so no number in the paper is typed by hand. --skip-weights drops the five weight-loading sections and finishes in about a minute.

Notes on the weights

• chains/ holds 112 files: SFT (8 checkpoints), PPO flexible (8), PPO strict (6), PPO high-KL (3), PPO shuffled-label (3), each at layers 6, 12, 18, 23. k=64 throughout except where layer 23 uses k=256.
• real_arm_L18/ stores bare decoder matrices ([7168, 896]), not full state dicts. These are what the pairwise direction test consumes.
• null_trajectories/ excludes the DONE marker files.
• Per-step weights exist only for the layer-18 TopK real-arm replication. The controlled rerun saved base SAEs alone, for both architectures, so controlled_rerun/ holds 9 base checkpoints (4 TopK, 5 JumpReLU) and no intermediate chain weights. Nothing is lost for verification: every per-step dec_cos, dead_frac, mean_l0 and kept_epoch is recorded in tables/controlled_rerun/*.csv, which is enough to reproduce both the raw and the dead-latent-adjusted shares. For example, at layer 18 the JumpReLU arm gives a raw share of 95.1% and 91.3% after dividing out frozen latents, both recomputable from that CSV alone.
• Activation tensors are not included; they are regenerable from the public base model and GSM8K.

Caveats we would rather state than have you find

• Layer 23 appears throughout the tables and is not read in the paper: its residual stream is mean-dominated and no SAE we trained reconstructs it acceptably.
• Seed replication covers the SAE fit, not the fine-tuning run. Every real chain descends from the same sequence of checkpoints.
• An earlier BatchTopK architecture grid is deliberately excluded: its threshold received no gradient, so it is not evidence and the paper does not use it.
• The strict chain's layer-23 k=256 robustness variant is not here. Layer 23 is not read in the paper and that variant backs no claim in it.
• The epoch-budget sweep weights are not here either; Appendix E.1's numbers are in tables/drift_out/.

Contents at a glance

Path	Files
weights/chains/	112
weights/null_step/	48 (4 canonical + 44 seeds)
weights/null_trajectories/	96 (4 layers x 3 seeds x 8 steps)
weights/real_arm_L18/	57 (base + 4 real x 7 + 4 null x 7)
weights/controlled_rerun/	9
weights/tuned_lens/	4
tables/	38
run_metadata/	26