Text Generation
Transformers
Safetensors
English
Japanese
llama
conversational
text-generation-inference
maym15 commited on
Commit
36f4273
·
verified ·
1 Parent(s): ed1411d

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +4 -0
README.md CHANGED
@@ -80,6 +80,8 @@ The website [https://swallow-llm.github.io/](https://swallow-llm.github.io/index
80
 
81
  |Model|JCom.|JEMHopQA|NIILC|JSQuAD|XL-Sum|MGSM|WMT20-en-ja|WMT20-ja-en|JMMLU|JHumanEval|Ja Avg|
82
  |---|---|---|---|---|---|---|---|---|---|---|---|
 
 
83
  | llm-jp-3-7.2b-instruct3 | 0.780 | 0.297 | 0.570 | 0.882 | 0.132 | 0.344 | 0.251 | 0.189 | 0.422 | 0.196 | 0.406 |
84
  | Qwen2-7B-Instruct | 0.888 | 0.390 | 0.379 | 0.897 | 0.126 | 0.576 | 0.206 | 0.190 | 0.571 | 0.555 | 0.478 |
85
  | Qwen2.5-7B-Instruct | 0.915 | 0.429 | 0.391 | 0.891 | 0.168 | 0.632 | 0.211 | 0.192 | 0.623 | 0.532 | 0.498 |
@@ -99,6 +101,8 @@ The website [https://swallow-llm.github.io/](https://swallow-llm.github.io/index
99
 
100
  |Model|OpenBookQA|TriviaQA|HellaSWAG|SQuAD2.0|XWINO|MMLU|GSM8K|BBH|HumanEval|En Avg|
101
  |---|---|---|---|---|---|---|---|---|---|---|
 
 
102
  | llm-jp-3-7.2b-instruct3 | 0.328 | 0.479 | 0.563 | 0.501 | 0.876 | 0.462 | 0.264 | 0.028 | 0.420 | 0.219 | 0.414 |
103
  | Qwen2-7B-Instruct | 0.396 | 0.547 | 0.615 | 0.593 | 0.886 | 0.707 | 0.626 | 0.504 | 0.304 | 0.643 | 0.582 |
104
  | Qwen2.5-7B-Instruct | 0.428 | 0.519 | 0.624 | 0.569 | 0.877 | 0.742 | 0.739 | 0.688 | 0.217 | 0.636 | 0.604 |
 
80
 
81
  |Model|JCom.|JEMHopQA|NIILC|JSQuAD|XL-Sum|MGSM|WMT20-en-ja|WMT20-ja-en|JMMLU|JHumanEval|Ja Avg|
82
  |---|---|---|---|---|---|---|---|---|---|---|---|
83
+ | |4-shot|4-shot|4-shot|4-shot|1-shot|4-shot|4-shot|4-shot|5-shot|0-shot| |
84
+ | |EM acc|Char-F1|Char-F1|Char-F1|ROUGE-2|EM acc|BLEU|BLEU|EM acc|pass@1| |
85
  | llm-jp-3-7.2b-instruct3 | 0.780 | 0.297 | 0.570 | 0.882 | 0.132 | 0.344 | 0.251 | 0.189 | 0.422 | 0.196 | 0.406 |
86
  | Qwen2-7B-Instruct | 0.888 | 0.390 | 0.379 | 0.897 | 0.126 | 0.576 | 0.206 | 0.190 | 0.571 | 0.555 | 0.478 |
87
  | Qwen2.5-7B-Instruct | 0.915 | 0.429 | 0.391 | 0.891 | 0.168 | 0.632 | 0.211 | 0.192 | 0.623 | 0.532 | 0.498 |
 
101
 
102
  |Model|OpenBookQA|TriviaQA|HellaSWAG|SQuAD2.0|XWINO|MMLU|GSM8K|BBH|HumanEval|En Avg|
103
  |---|---|---|---|---|---|---|---|---|---|---|
104
+ | |4-shot|4-shot|4-shot|4-shot|4-shot|5-shot|4-shot|3-shot|0-shot| |
105
+ | |Acc|EM acc|Acc|EM acc|Acc|Acc|EM acc|CoT EM Acc|pass@1| |
106
  | llm-jp-3-7.2b-instruct3 | 0.328 | 0.479 | 0.563 | 0.501 | 0.876 | 0.462 | 0.264 | 0.028 | 0.420 | 0.219 | 0.414 |
107
  | Qwen2-7B-Instruct | 0.396 | 0.547 | 0.615 | 0.593 | 0.886 | 0.707 | 0.626 | 0.504 | 0.304 | 0.643 | 0.582 |
108
  | Qwen2.5-7B-Instruct | 0.428 | 0.519 | 0.624 | 0.569 | 0.877 | 0.742 | 0.739 | 0.688 | 0.217 | 0.636 | 0.604 |