tim-lawson
commited on
Commit
•
e0392be
1
Parent(s):
a34dc66
Update README.md
Browse files
README.md
CHANGED
@@ -3,15 +3,21 @@ language: en
|
|
3 |
library_name: mlsae
|
4 |
license: mit
|
5 |
tags:
|
6 |
-
- model_hub_mixin
|
7 |
-
- pytorch_model_hub_mixin
|
8 |
datasets:
|
9 |
-
- monology/pile-uncopyrighted
|
10 |
---
|
11 |
|
12 |
-
|
13 |
-
For more details, see the [paper](https://arxiv.org/submit/5837813) and the [Weights & Biases project](https://wandb.ai/timlawson-/mlsae).
|
14 |
|
15 |
-
|
|
|
|
|
|
|
|
|
|
|
16 |
|
17 |
-
-
|
|
|
|
|
|
3 |
library_name: mlsae
|
4 |
license: mit
|
5 |
tags:
|
6 |
+
- model_hub_mixin
|
7 |
+
- pytorch_model_hub_mixin
|
8 |
datasets:
|
9 |
+
- monology/pile-uncopyrighted
|
10 |
---
|
11 |
|
12 |
+
# mlsae-pythia-70m-deduped-x1-k16
|
|
|
13 |
|
14 |
+
A Multi-Layer Sparse Autoencoder (MLSAE) trained on the residual stream
|
15 |
+
activation vectors from every layer of
|
16 |
+
[EleutherAI/pythia-70m-deduped](https://huggingface.co/EleutherAI/pythia-70m-deduped)
|
17 |
+
with an expansion factor of 1 and k = 16, over 1 billion tokens from
|
18 |
+
[monology/pile-uncopyrighted](https://huggingface.co/datasets/monology/pile-uncopyrighted).
|
19 |
+
For more details, see:
|
20 |
|
21 |
+
- Paper: <https://arxiv.org/abs/2409.04185>,
|
22 |
+
- GitHub repository: <https://github.com/tim-lawson/mlsae>
|
23 |
+
- Weights & Biases project: <https://wandb.ai/timlawson-/mlsae>
|