SAE on gLM2_650M layer 24 (intergenic tokens)
A BatchTopK sparse autoencoder trained on the residual stream of
tattabio/gLM2_650M layer 24. Full 4096-token
contigs are embedded, but only intergenic (nucleotide) positions are
trained on.
Configuration
| architecture | BatchTopK |
| k | 8 |
| d_in | 1280 |
| d_sae | 16384 (expansion 12x) |
| hook | glm2.encoder.layers.24 |
| base model | tattabio/gLM2_650M |
| context size | 4096 |
| training tokens | 1000.0M |
| trained on | intergenic (nucleotide) tokens |
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support