Boltz-1 SAEs — diffusion module
TopK sparse autoencoders trained on activations from Boltz-1's diffusion coordinate module, sampled at three points along the denoising trajectory. Published for anonymous double-blind review; no authorship or affiliation is attached to this repository.
Architecture
| SAE type | TopK, k = 256 |
| Latent width | 2048 |
| Input width | 768 |
| Weight L2 | 3e-3 |
| Preprocessing | training-set mean subtracted before encoding |
| Decoder | unit-normalised |
| Training steps | 500,000 |
| Seeds per layer | 3 |
| Base model | Boltz-1 |
Note the input width: 768 here, against 384 for the trunk repositories.
Contents
60 runs across three sampling steps, seven DiffusionTransformer layers and
three seeds. Coverage is not a full grid:
| sampling step | layers |
|---|---|
rec0 |
0, 2, 4, 10, 14, 18, 22 |
rec10 |
0, 2, 4, 14, 18, 22 |
rec50 |
0, 2, 4, 10, 14, 18, 22 |
Layer 10 is absent at step 10; the other two steps have it.
rec<STEP>/layer<L>/
diffusion<L>_topk256_lat2048_demean_longtrain500000_l2_3e-3_alive_cross_seed.json
diffusion<L>_topk256_lat2048_demean_longtrain500000_l2_3e-3_seed<S>/
checkpoint_step_500000.pt
config.json
mean_vector.npy
eval_step_500000.json
stats.jsonl
The rec<STEP> directory is the denoising step, not a recycle index. Each
config.json records it in its rec field, which the trunk repositories use
for the recycle iteration instead.
This extra top-level level is the one structural difference from the trunk
repositories, whose runs sit directly under layer<L>/.
Loading
from huggingface_hub import hf_hub_download
repo = "anonboltzinterp/Boltz1-SAEs-L2-Diffusion"
run = "rec0/layer22/diffusion22_topk256_lat2048_demean_longtrain500000_l2_3e-3_seed1"
for name in ("config.json", "mean_vector.npy", "checkpoint_step_500000.pt"):
hf_hub_download(repo_id=repo, filename=f"{run}/{name}", local_dir="sae")
Companion repositories
anonboltzinterp/Boltz1-SAEs-L2-rec1— Pairformer trunk SAEs at recycle 1anonboltzinterp/Boltz1-SAEs-L2-rec0— Pairformer trunk SAEs at recycle 0