Update README.md
Browse files
README.md
CHANGED
@@ -1,3 +1,31 @@
|
|
1 |
---
|
|
|
|
|
|
|
|
|
|
|
2 |
license: bigscience-openrail-m
|
3 |
---
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
1 |
---
|
2 |
+
language:
|
3 |
+
- en
|
4 |
+
tags:
|
5 |
+
- pytorch
|
6 |
+
- causal-lm
|
7 |
license: bigscience-openrail-m
|
8 |
---
|
9 |
+
|
10 |
+
|
11 |
+
GeoV-9B is a 20 billion parameter autoregressive language model
|
12 |
+
|
13 |
+
### Model details
|
14 |
+
|
15 |
+
- Developed by: [Georges Harik](http://twitter.com/gharik)
|
16 |
+
- Model type: Transformer-based Language Model
|
17 |
+
- Language: English
|
18 |
+
|
19 |
+
<figure style="width:30em">
|
20 |
+
|
21 |
+
| Hyperparameter | Value |
|
22 |
+
| ---------------------- | ----------- |
|
23 |
+
| n<sub>parameters</sub> | 9B |
|
24 |
+
| n<sub>layers</sub> | 32 |
|
25 |
+
| d<sub>model</sub> | 5120 |
|
26 |
+
| n<sub>heads</sub> | 40 |
|
27 |
+
| d<sub>head</sub> | 128 |
|
28 |
+
| n<sub>vocab</sub> | 65500 |
|
29 |
+
| Sequence Length | 2049 |
|
30 |
+
</figure>
|
31 |
+
|