Update README.md
Browse files
README.md
CHANGED
|
@@ -26,6 +26,20 @@ Please read the existing GPT-NeoX-20B documentation before asking about the mode
|
|
| 26 |
on Discord. For general correspondence: [contact@eleuther.
|
| 27 |
ai](mailto:contact@eleuther.ai).
|
| 28 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 29 |
### Uses and limitations
|
| 30 |
|
| 31 |
#### Intended use
|
|
|
|
| 26 |
on Discord. For general correspondence: [contact@eleuther.
|
| 27 |
ai](mailto:contact@eleuther.ai).
|
| 28 |
|
| 29 |
+
<figure style="width:30em">
|
| 30 |
+
|
| 31 |
+
| Hyperparameter | Value |
|
| 32 |
+
| -------------------------- | ----------- |
|
| 33 |
+
| n<sub>parameters</sub> | |
|
| 34 |
+
| n<sub>encoder layers</sub> | 12 |
|
| 35 |
+
| n<sub>decoder layers</sub> | 12 |
|
| 36 |
+
| d<sub>model</sub> | 2048 |
|
| 37 |
+
| n<sub>heads</sub> | 12 |
|
| 38 |
+
| d<sub>head</sub> | 64 |
|
| 39 |
+
| n<sub>vocab</sub> | 32128 |
|
| 40 |
+
| Sequence Length | 512 |
|
| 41 |
+
</figure>
|
| 42 |
+
|
| 43 |
### Uses and limitations
|
| 44 |
|
| 45 |
#### Intended use
|