Update README.md
Browse files
README.md
CHANGED
@@ -5,7 +5,11 @@ tags: []
|
|
5 |
|
6 |
# Model Card for Model ID
|
7 |
|
8 |
-
Trained on 100B tokens.
|
|
|
|
|
|
|
|
|
9 |
|
10 |
|
11 |
## Model Details
|
|
|
5 |
|
6 |
# Model Card for Model ID
|
7 |
|
8 |
+
Trained on 100B tokens.
|
9 |
+
- 1e-3 LR
|
10 |
+
- 0.1 wd
|
11 |
+
- WSD scheduler with 10% decay
|
12 |
+
- 80% code, 10% NL, 10% instruction data
|
13 |
|
14 |
|
15 |
## Model Details
|