Sparse Autoencoders
Collection
SAEs are tools for understanding the internal representations of neural networks. These can be loaded using https://github.com/EleutherAI/sae
•
7 items
•
Updated
SAEs and transcoders can be loaded using https://github.com/EleutherAI/sae.
These transcoders were trained on the outputs of the first 15 MLPs in deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B. We used 10 billion tokens from FineWeb edu deduped at a context length of 2048. The number of latents is 65,536 and a linear skip connection is included.
Fraction of variance unexplained ranges from 0.01 to 0.37.
- Downloads last month
- 15
Inference Providers
NEW
This model is not currently available via any of the supported third-party Inference Providers, and
HF Inference API was unable to determine this model's library.
Model tree for EleutherAI/skip-transcoder-DeepSeek-R1-Distill-Qwen-1.5B-65k
Base model
deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B