Sparse Autoencoders
Collection
SAE is an approach for extracting the internal representations of neural networks. • 24 items • Updated
This is a set of transcoders trained on the TinyStories-3M using the 2.2M sample of the TinyStories dataset, which comes out to roughly 2.2B tokens using the full GPT-Neo tokenizer. We trained the transcoders on the MLP layers 2, 4, and 7 with a dictionary size twice that of the MLP layer. The transcoders are organized by hookpoint.
Usage
With the sae library installed, you can access an SAE like this:
from sae import Sae
sae = Sae.load_from_hub("abmakkeh/transcoder-TinyStories-3M-2x", hookpoint="h.1.mlp")
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support
Model tree for abmakkeh/transcoder-TinyStories-3M-2x
Base model
roneneldan/TinyStories-3M