Hugging Face's logo

tokyotech-llm
/
Llama-3.1-Swallow-70B-v0.1

Text Generation
Transformers
Safetensors
English
Japanese
llama
text-generation-inference
Inference Endpoints
Model card Files Community
Taishi-N324 commited on
Commit
d21580c
·
verified ·
1 Parent(s): 5172e20

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +6 -6
README.md CHANGED
@@ -60,14 +60,14 @@ This repository provides large language models developed by [Swallow-LLM](https:
60
  |---|---|---|---|---|---|---|---|---|---|---|
61
  | |4-shot|4-shot|4-shot|4-shot|4-shot|5-shot|4-shot|3-shot|0-shot| |
62
  | |Acc|EM acc|Acc|EM acc|Acc|Acc|EM acc|CoT EM Acc|pass@1| |
63
- | Qwen2-72B | 0.4160 | **0.7890** | 0.6766 | **0.4052** | 0.9161 | 0.8428 | **0.8908** | 0.6388 | **0.6049** | **0.6867** |
64
- | Qwen2.5-72B | 0.4160 | 0.7604 | **0.6849** | 0.3997 | 0.9015 | **0.8608** | 0.8726 | **0.7268** | 0.5543 | 0.6863 |
65
- | Sarashina2-70B | 0.3920 | 0.5373 | 0.6270 | 0.4174 | **0.9178** | 0.6303 | 0.0106 | 0.6386 | 0.2799 | 0.4945 |
66
- | Llama 3 70B | 0.4360 | 0.8263 | 0.6909 | 0.4071 | 0.9213 | 0.7870 | 0.8014 | 0.8266 | 0.5177 | 0.6905 |
67
  | Llama 3.1 70B | **0.4420** | 0.8288 | 0.6898 | 0.4050 | 0.9196 | 0.7846 | 0.7991 | 0.6566 | 0.5476 | 0.6748 |
68
- | Llama 3 Youko 70B | 0.4300 | 0.8291 | 0.6900 | 0.4057 | 0.9222 | 0.7862 | 0.7968 | 0.8275 | 0.4128 | 0.6778 |
69
  | Llama 3 Swallow 70B | 0.4240 | 0.8231 | 0.6828 | 0.4059 | 0.9234 | 0.7745 | 0.8143 | 0.7352 | 0.4909 | 0.6749 |
70
- | Llama 3.1 Swallow 70B | 0.4320 | 0.8262 | 0.6898 | 0.4018 | 0.9277 | 0.7724 | 0.8089 | 0.8063 | 0.5396 | 0.6894 |
71
 
72
  ## Evaluation Benchmarks
73
 
 
60
  |---|---|---|---|---|---|---|---|---|---|---|
61
  | |4-shot|4-shot|4-shot|4-shot|4-shot|5-shot|4-shot|3-shot|0-shot| |
62
  | |Acc|EM acc|Acc|EM acc|Acc|Acc|EM acc|CoT EM Acc|pass@1| |
63
+ | Qwen2-72B | 0.4160 | 0.7890 | 0.6766 | 0.4052 | 0.9161 | 0.8428 | **0.8908** | 0.6388 | **0.6049** | **0.6867** |
64
+ | Qwen2.5-72B | 0.4160 | 0.7604 | 0.6849 | 0.3997 | 0.9015 | **0.8608** | 0.8726 | **0.7268** | 0.5543 | 0.6863 |
65
+ | Sarashina2-70B | 0.3920 | 0.5373 | 0.6270 | **0.4174** | 0.9178 | 0.6303 | 0.0106 | 0.6386 | 0.2799 | 0.4945 |
66
+ | Llama 3 70B | 0.4360 | 0.8263 | **0.6909** | 0.4071 | 0.9213 | 0.7870 | 0.8014 | 0.8266 | 0.5177 | 0.6905 |
67
  | Llama 3.1 70B | **0.4420** | 0.8288 | 0.6898 | 0.4050 | 0.9196 | 0.7846 | 0.7991 | 0.6566 | 0.5476 | 0.6748 |
68
+ | Llama 3 Youko 70B | 0.4300 | **0.8291** | 0.6900 | 0.4057 | 0.9222 | 0.7862 | 0.7968 | 0.8275 | 0.4128 | 0.6778 |
69
  | Llama 3 Swallow 70B | 0.4240 | 0.8231 | 0.6828 | 0.4059 | 0.9234 | 0.7745 | 0.8143 | 0.7352 | 0.4909 | 0.6749 |
70
+ | Llama 3.1 Swallow 70B | 0.4320 | 0.8262 | 0.6898 | 0.4018 | **0.9277** | 0.7724 | 0.8089 | 0.8063 | 0.5396 | 0.6894 |
71
 
72
  ## Evaluation Benchmarks
73