Update README.md
Browse files
README.md
CHANGED
@@ -14,11 +14,9 @@ This is a landing page for **Gemma Scope**, a comprehensive, open suite of spars
|
|
14 |
- https://huggingface.co/google/gemma-scope-2b-pt-res
|
15 |
- https://huggingface.co/google/gemma-scope-2b-pt-mlp
|
16 |
- https://huggingface.co/google/gemma-scope-2b-pt-att
|
17 |
-
- https://huggingface.co/google/gemma-scope-2b-pt-transcoders
|
18 |
- https://huggingface.co/google/gemma-scope-9b-pt-res
|
19 |
- https://huggingface.co/google/gemma-scope-9b-pt-mlp
|
20 |
- https://huggingface.co/google/gemma-scope-9b-pt-att
|
21 |
-
- https://huggingface.co/google/gemma-scope-9b-it-res
|
22 |
- https://huggingface.co/google/gemma-scope-27b-pt-res
|
23 |
|
24 |
# Key links:
|
@@ -35,7 +33,7 @@ The full list of SAEs we trained at which sites and layers are linked from the f
|
|
35 |
|
36 |
| <big>Gemma 2 Model</big> | <big>SAE Width</big> | <big>Attention</big> | <big>MLP</big> | <big>Residual</big> | <big>Tokens</big> |
|
37 |
|---------------|-----------|-----------|-----|----------|----------|
|
38 |
-
| 2.6B PT<br>(26 layers) | 2^14 ≈ 16.4K | [All](https://huggingface.co/google/gemma-scope-2b-pt-att) | [All](https://huggingface.co/google/gemma-scope-2b-pt-mlp)
|
39 |
| | 2^15 | | | {[12](https://huggingface.co/google/gemma-scope-2b-pt-res/tree/main/layer_12/width_32k/)} | 8B |
|
40 |
| | 2^16 | [All](https://huggingface.co/google/gemma-scope-2b-pt-att) | [All](https://huggingface.co/google/gemma-scope-2b-pt-mlp) | [All](https://huggingface.co/google/gemma-scope-2b-pt-res) | 8B |
|
41 |
| | 2^17 | | | {[12](https://huggingface.co/google/gemma-scope-2b-pt-res/tree/main/layer_12/width_131k/)} | 8B |
|
@@ -50,5 +48,3 @@ The full list of SAEs we trained at which sites and layers are linked from the f
|
|
50 |
| | 2^19 | | | {[20](https://huggingface.co/google/gemma-scope-9b-pt-res/tree/main/layer_20/width_524k/)} | 8B |
|
51 |
| | 2^20 | | | {[9](https://huggingface.co/google/gemma-scope-9b-pt-res/tree/main/layer_9/width_1m/), [20](https://huggingface.co/google/gemma-scope-9b-pt-res/tree/main/layer_20/width_1m/), [31](https://huggingface.co/google/gemma-scope-9b-pt-res/tree/main/layer_31/width_1m/)} | 16B |
|
52 |
| 27B PT<br>(46 layers) | 2^17 | | | {[10](https://huggingface.co/google/gemma-scope-27b-pt-res/tree/main/layer_10/width_131k/), [22](https://huggingface.co/google/gemma-scope-27b-pt-res/tree/main/layer_22/width_131k/), [34](https://huggingface.co/google/gemma-scope-27b-pt-res/tree/main/layer_34/width_131k/)} | 8B |
|
53 |
-
| 9B IT<br>(42 layers) | 2^14 | | | {[9](https://huggingface.co/google/gemma-scope-9b-it-res/tree/main/layer_9/width_16k/), [20](https://huggingface.co/google/gemma-scope-9b-it-res/tree/main/layer_20/width_16k/), [31](https://huggingface.co/google/gemma-scope-9b-it-res/tree/main/layer_31/width_16k/)} | 4B |
|
54 |
-
| | 2^17 | | | {[9](https://huggingface.co/google/gemma-scope-9b-it-res/tree/main/layer_9/width_131k/), [20](https://huggingface.co/google/gemma-scope-9b-it-res/tree/main/layer_20/width_131k/), [31](https://huggingface.co/google/gemma-scope-9b-it-res/tree/main/layer_31/width_131k/)} | 8B |
|
|
|
14 |
- https://huggingface.co/google/gemma-scope-2b-pt-res
|
15 |
- https://huggingface.co/google/gemma-scope-2b-pt-mlp
|
16 |
- https://huggingface.co/google/gemma-scope-2b-pt-att
|
|
|
17 |
- https://huggingface.co/google/gemma-scope-9b-pt-res
|
18 |
- https://huggingface.co/google/gemma-scope-9b-pt-mlp
|
19 |
- https://huggingface.co/google/gemma-scope-9b-pt-att
|
|
|
20 |
- https://huggingface.co/google/gemma-scope-27b-pt-res
|
21 |
|
22 |
# Key links:
|
|
|
33 |
|
34 |
| <big>Gemma 2 Model</big> | <big>SAE Width</big> | <big>Attention</big> | <big>MLP</big> | <big>Residual</big> | <big>Tokens</big> |
|
35 |
|---------------|-----------|-----------|-----|----------|----------|
|
36 |
+
| 2.6B PT<br>(26 layers) | 2^14 ≈ 16.4K | [All](https://huggingface.co/google/gemma-scope-2b-pt-att) | [All](https://huggingface.co/google/gemma-scope-2b-pt-mlp) | [All](https://huggingface.co/google/gemma-scope-2b-pt-res) | 4B |
|
37 |
| | 2^15 | | | {[12](https://huggingface.co/google/gemma-scope-2b-pt-res/tree/main/layer_12/width_32k/)} | 8B |
|
38 |
| | 2^16 | [All](https://huggingface.co/google/gemma-scope-2b-pt-att) | [All](https://huggingface.co/google/gemma-scope-2b-pt-mlp) | [All](https://huggingface.co/google/gemma-scope-2b-pt-res) | 8B |
|
39 |
| | 2^17 | | | {[12](https://huggingface.co/google/gemma-scope-2b-pt-res/tree/main/layer_12/width_131k/)} | 8B |
|
|
|
48 |
| | 2^19 | | | {[20](https://huggingface.co/google/gemma-scope-9b-pt-res/tree/main/layer_20/width_524k/)} | 8B |
|
49 |
| | 2^20 | | | {[9](https://huggingface.co/google/gemma-scope-9b-pt-res/tree/main/layer_9/width_1m/), [20](https://huggingface.co/google/gemma-scope-9b-pt-res/tree/main/layer_20/width_1m/), [31](https://huggingface.co/google/gemma-scope-9b-pt-res/tree/main/layer_31/width_1m/)} | 16B |
|
50 |
| 27B PT<br>(46 layers) | 2^17 | | | {[10](https://huggingface.co/google/gemma-scope-27b-pt-res/tree/main/layer_10/width_131k/), [22](https://huggingface.co/google/gemma-scope-27b-pt-res/tree/main/layer_22/width_131k/), [34](https://huggingface.co/google/gemma-scope-27b-pt-res/tree/main/layer_34/width_131k/)} | 8B |
|
|
|
|