tim-lawson commited on
Commit
e0392be
1 Parent(s): a34dc66

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +13 -7
README.md CHANGED
@@ -3,15 +3,21 @@ language: en
3
  library_name: mlsae
4
  license: mit
5
  tags:
6
- - model_hub_mixin
7
- - pytorch_model_hub_mixin
8
  datasets:
9
- - monology/pile-uncopyrighted
10
  ---
11
 
12
- A Multi-Layer Sparse Autoencoder (MLSAE) trained on [EleutherAI/pythia-70m-deduped](https://huggingface.co/EleutherAI/pythia-70m-deduped) and [monology/pile-uncopyrighted](https://huggingface.co/datasets/monology/pile-uncopyrighted), with an expansion factor of 1 and $k = 16$.
13
- For more details, see the [paper](https://arxiv.org/submit/5837813) and the [Weights & Biases project](https://wandb.ai/timlawson-/mlsae).
14
 
15
- This model has been pushed to the Hub using the [PytorchModelHubMixin](https://huggingface.co/docs/huggingface_hub/package_reference/mixins#huggingface_hub.PyTorchModelHubMixin) integration:
 
 
 
 
 
16
 
17
- - Library: <https://github.com/tim-lawson/mlsae>
 
 
 
3
  library_name: mlsae
4
  license: mit
5
  tags:
6
+ - model_hub_mixin
7
+ - pytorch_model_hub_mixin
8
  datasets:
9
+ - monology/pile-uncopyrighted
10
  ---
11
 
12
+ # mlsae-pythia-70m-deduped-x1-k16
 
13
 
14
+ A Multi-Layer Sparse Autoencoder (MLSAE) trained on the residual stream
15
+ activation vectors from every layer of
16
+ [EleutherAI/pythia-70m-deduped](https://huggingface.co/EleutherAI/pythia-70m-deduped)
17
+ with an expansion factor of 1 and k = 16, over 1 billion tokens from
18
+ [monology/pile-uncopyrighted](https://huggingface.co/datasets/monology/pile-uncopyrighted).
19
+ For more details, see:
20
 
21
+ - Paper: <https://arxiv.org/abs/2409.04185>,
22
+ - GitHub repository: <https://github.com/tim-lawson/mlsae>
23
+ - Weights & Biases project: <https://wandb.ai/timlawson-/mlsae>