Update README.md
Browse files
README.md
CHANGED
|
@@ -4,7 +4,6 @@ extra_gated_button_content: Submit
|
|
| 4 |
extra_gated_fields:
|
| 5 |
I agree to share my name, email address and username: checkbox
|
| 6 |
I confirm that I understand this project is for research purposes only, and confirm that I agree to follow the LICENSE of this model: checkbox
|
| 7 |
-
I confirm that THIS version of repo is UNDER DEVELOPMENT(Alpha release will come within ~1 week): checkbox
|
| 8 |
language:
|
| 9 |
- en
|
| 10 |
- ko
|
|
@@ -18,9 +17,8 @@ tags:
|
|
| 18 |
library_name: transformers
|
| 19 |
---
|
| 20 |
|
| 21 |
-
> 🚧 Note: this repo is under construction, will release first version within ~1 week 🚧
|
| 22 |
|
| 23 |
-
> Update @ 2023.
|
| 24 |
|
| 25 |
# **beomi/Yi-Ko-6B**
|
| 26 |
|
|
@@ -49,7 +47,7 @@ Yi-Ko series models are an auto-regressive language model that uses an optimized
|
|
| 49 |
|
| 50 |
|Model Name|Training Data|Params|Context Length|GQA|Trained Tokens|LR|Batch Size(per step)|
|
| 51 |
|---|---|---|---|---|---|---|---|
|
| 52 |
-
|Yi-Ko-6B|*A mix of Korean + English online data*|6B|4k|O|
|
| 53 |
|
| 54 |
**Vocab Expansion**
|
| 55 |
|
|
|
|
| 4 |
extra_gated_fields:
|
| 5 |
I agree to share my name, email address and username: checkbox
|
| 6 |
I confirm that I understand this project is for research purposes only, and confirm that I agree to follow the LICENSE of this model: checkbox
|
|
|
|
| 7 |
language:
|
| 8 |
- en
|
| 9 |
- ko
|
|
|
|
| 17 |
library_name: transformers
|
| 18 |
---
|
| 19 |
|
|
|
|
| 20 |
|
| 21 |
+
> Update @ 2023.12.01 Alpha Release of Yi-Ko(KoEN)-6B model 🎉
|
| 22 |
|
| 23 |
# **beomi/Yi-Ko-6B**
|
| 24 |
|
|
|
|
| 47 |
|
| 48 |
|Model Name|Training Data|Params|Context Length|GQA|Trained Tokens|LR|Batch Size(per step)|
|
| 49 |
|---|---|---|---|---|---|---|---|
|
| 50 |
+
|Yi-Ko-6B|*A mix of Korean + English online data*|6B|4k|O|>60B|5e<sup>-5</sup>|2048|
|
| 51 |
|
| 52 |
**Vocab Expansion**
|
| 53 |
|