Hugging Face's logo

PracticeLLM
/
Custom-KoLLM-13B-v2

Text Generation
Transformers
PyTorch
Korean
llama
Inference Endpoints
text-generation-inference
Model card Files Community
kyujinpy commited on
Commit
64b72e9
•
1 Parent(s): 125f10f

Upload README.md

Browse files
Files changed (1) hide show
  1. README.md +13 -1
README.md CHANGED
@@ -30,8 +30,20 @@ license: cc-by-nc-sa-4.0
30
  | Model | Average | Ko-ARC | Ko-HellaSwag | Ko-MMLU | Ko-TruthfulQA | Ko-CommonGen V2 |
31
  | --- | --- | --- | --- | --- | --- | --- |
32
  | ⭐My custom LLM 13B-v1⭐ | 50.19 | 45.99 | 56.93 | 41.78 | 41.66 | **64.58** |
33
- | **⭐My custom LLM 13B v2⭐** | 48.28 | 45.73 | 56.97 | 38.77 | 38.75 | 61.16 |
34
 
 
 
 
 
 
 
 
 
 
 
 
 
35
  # Implementation Code
36
  ```python
37
  ### KO-Platypus
 
30
  | Model | Average | Ko-ARC | Ko-HellaSwag | Ko-MMLU | Ko-TruthfulQA | Ko-CommonGen V2 |
31
  | --- | --- | --- | --- | --- | --- | --- |
32
  | ⭐My custom LLM 13B-v1⭐ | 50.19 | 45.99 | 56.93 | 41.78 | 41.66 | **64.58** |
33
+ | **⭐My custom LLM 13B-v2⭐** | 48.28 | 45.73 | 56.97 | 38.77 | 38.75 | 61.16 |
34
 
35
+ ---
36
+ # Model comparisons2
37
+ > AI-Harness evaluation; [link](https://github.com/Beomi/ko-lm-evaluation-harness)
38
+
39
+ | Model | Copa | Copa | HellaSwag | HellaSwag | BoolQ | BoolQ | Sentineg | Sentineg |
40
+ | --- | --- | --- | --- | --- | --- | --- | --- | --- |
41
+ | | 0-shot | 5-shot | 0-shot | 5-shot | 0-shot | 5-shot | 0-shot | 5-shot |
42
+ | ⭐My custom LLM 13B-v1⭐ | 0.7987 | 0.8269 | 0.4994 | 0.5660 | 0.3343 | 0.5060 | 0.6984 | 0.9723 |
43
+ | **⭐My custom LLM 13B-v2⭐** | 0.7938 | 0.8209 | 0.4978 | 0.4893 | 0.3343 | 0.5614 | 0.6283 | 0.9773 |
44
+ | [beomi/llama-2-koen-13b](https://huggingface.co/beomi/llama-2-koen-13b) | 0.7768 | 0.8128 | 0.4999 | 0.5127 | 0.3988 | 0.7038 | 0.5870 | 0.9748 |
45
+
46
+ ---
47
  # Implementation Code
48
  ```python
49
  ### KO-Platypus