davidkim205
commited on
Commit
•
3a73f4d
1
Parent(s):
611f448
Update README.md
Browse files
README.md
CHANGED
@@ -6,23 +6,35 @@ tags: []
|
|
6 |
|
7 |
# Nox
|
8 |
|
9 |
-
|
10 |
-
|
11 |
-
## Features
|
12 |
-
- Supports fullfinetune, lora, qlora
|
13 |
-
- Supports Supervised Fine-Tuning, DPO Training
|
14 |
-
- High-quality ko dataset
|
15 |
-
- Ko Evaluation dataset and code
|
16 |
-
|
17 |
|
|
|
|
|
|
|
18 |
|
19 |
Currently, we are planning to make all code and datasets public.
|
20 |
|
21 |
Through this, users are expected to be able to freely conduct research and development using Nox.
|
22 |
|
23 |
-
I'll be back soon.
|
24 |
|
25 |
|
26 |
## Model Details
|
27 |
|
28 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
6 |
|
7 |
# Nox
|
8 |
|
9 |
+
![image/jpeg](https://cdn-uploads.huggingface.co/production/uploads/64241c3d774cc340797429fc/u60xmpVM1K0AtPXZtw8b4.jpeg)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
10 |
|
11 |
+
The nox project is a set of tools that make it easy to use various fine tuning technologies using solar models.
|
12 |
+
We constructed ko data using grammatically accurate data.(It's not perfect, but I tried my best.)
|
13 |
+
And we created nox-solar model using a fine-tuning technique(sft,dpo) Our model, the nox-solar model, ranked first on the [Open Ko-LLM Leaderboard](https://huggingface.co/spaces/upstage/open-ko-llm-leaderboard).
|
14 |
|
15 |
Currently, we are planning to make all code and datasets public.
|
16 |
|
17 |
Through this, users are expected to be able to freely conduct research and development using Nox.
|
18 |
|
|
|
19 |
|
20 |
|
21 |
## Model Details
|
22 |
|
23 |
+
* **Model Developers** : davidkim(changyeon kim)
|
24 |
+
* **Repository** : https://github.com/davidkim205/nox(will be updated soon.)
|
25 |
+
* **base mode** : Edentns/DataVortexS-10.7B-dpo-v1.11
|
26 |
+
* **sft dataset** : komt-124k(will be updated soon.)
|
27 |
+
* **dpo dataset** : comparison_v2_289k(will be updated soon.)
|
28 |
+
* **evalution** : [kollm_evalution](https://github.com/davidkim205/kollm_evaluation)
|
29 |
+
* **evalution dataset** : [open-ko-llm-leaderboard datasets](https://huggingface.co/collections/davidkim205/open-ko-llm-leaderboard-datasets-65eea9e87fc3ae80787ee15a)
|
30 |
+
|
31 |
+
## Evaluation
|
32 |
+
### [The Open Ko-LLM Leaderboard](https://huggingface.co/spaces/upstage/open-ko-llm-leaderboard)
|
33 |
+
| Model | Average | Ko-ARC | Ko-HellaSwag | Ko-MMLU | Ko-TruthfulQA | Ko-CommonGen V2 |
|
34 |
+
| ------------------------------ | ------- | ------ | ------------ | ------- | ------------- | --------------- |
|
35 |
+
| davidkim205/nox-solar-10.7b-v4 | 67.77 | 73.55 | 72.07 | 57.93 | 79.32 | 55.96 |
|
36 |
+
|
37 |
+
|
38 |
+
|
39 |
+
### [kollm_evalution](https://github.com/davidkim205/kollm_evalution)
|
40 |
+
(will be updated soon.)
|