Update README.md
Browse files
README.md
CHANGED
|
@@ -69,6 +69,7 @@ Details:
|
|
| 69 |
- 768 hidden size, 12 layers
|
| 70 |
- no MEGA chunking, 4096 context length
|
| 71 |
- EMA dimension 16, shared dimension 192
|
|
|
|
| 72 |
- train-from-scratch
|
| 73 |
|
| 74 |
|
|
|
|
| 69 |
- 768 hidden size, 12 layers
|
| 70 |
- no MEGA chunking, 4096 context length
|
| 71 |
- EMA dimension 16, shared dimension 192
|
| 72 |
+
- tokenizer: GPT NeoX
|
| 73 |
- train-from-scratch
|
| 74 |
|
| 75 |
|