---
license: apache-2.0
datasets:
- davidkim205/kollm-converations
language:
- ko
- en
library_name: transformers
---

- just for practice
- trained by transformers, peft, bitsandbytes
- evaluated by https://github.com/davidkim205/kollm_evaluation

|     Tasks      |Version|Filter|n-shot| Metric |Value |   |Stderr|
|----------------|------:|------|-----:|--------|-----:|---|------|
|kobest_wic      |      1|none  |     0|acc     |0.5056|±  |0.0141|
|                |       |none  |     0|f1      |0.4170|±  |N/A   |
|kobest_sentineg |      1|none  |     0|acc     |0.8791|±  |0.0164|
|                |       |none  |     0|f1      |0.8779|±  |N/A   |
|kobest_hellaswag|      1|none  |     0|acc     |0.4540|±  |0.0223|
|                |       |none  |     0|f1      |0.4522|±  |N/A   |
|                |       |none  |     0|acc_norm|0.5600|±  |0.0222|
|kobest_copa     |      1|none  |     0|acc     |0.6540|±  |0.0151|
|                |       |none  |     0|f1      |0.6535|±  |N/A   |
|kobest_boolq    |      1|none  |     0|acc     |0.5135|±  |0.0133|
|                |       |none  |     0|f1      |0.3592|±  |N/A   |
|ko_truthfulqa   |      2|none  |     0|acc     |0.2938|±  |0.0159|
|ko_hellaswag    |      1|none  |     0|acc     |0.3816|±  |0.0048|
|                |       |none  |     0|acc_norm|0.5171|±  |0.0050|
|ko_common_gen   |      1|none  |     0|acc     |0.8095|±  |0.0100|
|                |       |none  |     0|acc_norm|0.8095|±  |0.0100|
|ko_arc_easy     |      1|none  |     0|acc     |0.3311|±  |0.0138|
|                |       |none  |     0|acc_norm|0.4053|±  |0.0143|