---
license: apache-2.0
language:
- ko
- en
library_name: transformers
datasets:
- davidkim205/kollm-converations
- davidkim205/kollm-comparision
---

just for practice
- trained by transformers, peft, bitsandbytes
- evaluated by https://github.com/davidkim205/kollm_evaluation

base model
- beomi/OPEN-SOLAR-KO-10.7B

|     Tasks      |Version|Filter|n-shot| Metric |Value |   |Stderr|
|----------------|------:|------|-----:|--------|-----:|---|------|
|kobest_wic      |      1|none  |     0|acc     |0.4873|±  |0.0141|
|                |       |none  |     0|f1      |0.3276|±  |N/A   |
|kobest_sentineg |      1|none  |     0|acc     |0.6499|±  |0.0240|
|                |       |none  |     0|f1      |0.6134|±  |N/A   |
|kobest_hellaswag|      1|none  |     0|acc     |0.4980|±  |0.0224|
|                |       |none  |     0|f1      |0.4952|±  |N/A   |
|                |       |none  |     0|acc_norm|0.6280|±  |0.0216|
|kobest_copa     |      1|none  |     0|acc     |0.8030|±  |0.0126|
|                |       |none  |     0|f1      |0.8026|±  |N/A   |
|kobest_boolq    |      1|none  |     0|acc     |0.8711|±  |0.0089|
|                |       |none  |     0|f1      |0.8710|±  |N/A   |
|ko_truthfulqa   |      2|none  |     0|acc     |0.2717|±  |0.0156|
|ko_hellaswag    |      1|none  |     0|acc     |0.4109|±  |0.0049|
|                |       |none  |     0|acc_norm|0.5503|±  |0.0050|
|ko_common_gen   |      1|none  |     0|acc     |0.9041|±  |0.0075|
|                |       |none  |     0|acc_norm|0.9041|±  |0.0075|
|ko_arc_easy     |      1|none  |     0|acc     |0.3490|±  |0.0139|
|                |       |none  |     0|acc_norm|0.4326|±  |0.0145|