metadata
license: apache-2.0
datasets:
- davidkim205/kollm-converations
language:
- ko
- en
library_name: transformers
- just for practice
- trained by transformers, peft, bitsandbytes
- evaluated by https://github.com/davidkim205/kollm_evaluation
| Tasks | Version | Filter | n-shot | Metric | Value | Stderr | |
|---|---|---|---|---|---|---|---|
| kobest_wic | 1 | none | 0 | acc | 0.5056 | ± | 0.0141 |
| none | 0 | f1 | 0.4170 | ± | N/A | ||
| kobest_sentineg | 1 | none | 0 | acc | 0.8791 | ± | 0.0164 |
| none | 0 | f1 | 0.8779 | ± | N/A | ||
| kobest_hellaswag | 1 | none | 0 | acc | 0.4540 | ± | 0.0223 |
| none | 0 | f1 | 0.4522 | ± | N/A | ||
| none | 0 | acc_norm | 0.5600 | ± | 0.0222 | ||
| kobest_copa | 1 | none | 0 | acc | 0.6540 | ± | 0.0151 |
| none | 0 | f1 | 0.6535 | ± | N/A | ||
| kobest_boolq | 1 | none | 0 | acc | 0.5135 | ± | 0.0133 |
| none | 0 | f1 | 0.3592 | ± | N/A | ||
| ko_truthfulqa | 2 | none | 0 | acc | 0.2938 | ± | 0.0159 |
| ko_hellaswag | 1 | none | 0 | acc | 0.3816 | ± | 0.0048 |
| none | 0 | acc_norm | 0.5171 | ± | 0.0050 | ||
| ko_common_gen | 1 | none | 0 | acc | 0.8095 | ± | 0.0100 |
| none | 0 | acc_norm | 0.8095 | ± | 0.0100 | ||
| ko_arc_easy | 1 | none | 0 | acc | 0.3311 | ± | 0.0138 |
| none | 0 | acc_norm | 0.4053 | ± | 0.0143 |