Update README.md
Browse files
README.md
CHANGED
|
@@ -16,7 +16,7 @@ For further details or testing our instruct model, see [our demo site](https://p
|
|
| 16 |
- The model has been extended to a context length of 32K with position interpolation
|
| 17 |
- Checkpoint: 100 000 steps
|
| 18 |
|
| 19 |
-
## Dataset for
|
| 20 |
|
| 21 |
- Hungarian: 7.9 billion words, documents (763K) that exceed 5000 words in length
|
| 22 |
- English: Long Context QA (2 billion words), BookSum (78 million words)
|
|
|
|
| 16 |
- The model has been extended to a context length of 32K with position interpolation
|
| 17 |
- Checkpoint: 100 000 steps
|
| 18 |
|
| 19 |
+
## Dataset for continued pretraining
|
| 20 |
|
| 21 |
- Hungarian: 7.9 billion words, documents (763K) that exceed 5000 words in length
|
| 22 |
- English: Long Context QA (2 billion words), BookSum (78 million words)
|