Update README.md
Browse files
README.md
CHANGED
|
@@ -10,6 +10,7 @@ Trained on 100B tokens.
|
|
| 10 |
- 0.1 wd
|
| 11 |
- WSD scheduler with 10% decay
|
| 12 |
- 80% code, 10% NL, 10% instruction data
|
|
|
|
| 13 |
- 8x3090s 110~ hours
|
| 14 |
|
| 15 |
|
|
|
|
| 10 |
- 0.1 wd
|
| 11 |
- WSD scheduler with 10% decay
|
| 12 |
- 80% code, 10% NL, 10% instruction data
|
| 13 |
+
- Dataset decontaminated against popular benchmarks following [bigcode](https://github.com/bigcode-project/bigcode-dataset/tree/main/decontamination)
|
| 14 |
- 8x3090s 110~ hours
|
| 15 |
|
| 16 |
|