Commit
•
ca92d44
1
Parent(s):
bb742d2
update readme
Browse files
README.md
CHANGED
|
@@ -1,3 +1,6 @@
|
|
| 1 |
-
---
|
| 2 |
-
license: mit
|
| 3 |
-
---
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
license: mit
|
| 3 |
+
---
|
| 4 |
+
This contains the weights of a sparse autoencoder I trained on the residual activations of [Mistral-7B-Instruct-v0.1](https://huggingface.co/mistralai/Mistral-7B-Instruct-v0.1). I used [The Pile (uncopyrighted)](https://huggingface.co/datasets/monology/pile-uncopyrighted) for the training data. As of right now, I have only trained a single SAE (on layer 16), though I may do more in the future.
|
| 5 |
+
|
| 6 |
+
Here is the associated [GitHub repo](https://github.com/tylercosgrove/sparse-autoencoder-mistral7b) I used to train it, which you can also use to do some crude inference.
|