Hugging Face's logo

MediaTek-Research
/
Breeze-7B-Instruct-v0_1

Text Generation
Transformers
PyTorch
Safetensors
Chinese
English
mistral
conversational
text-generation-inference
Inference Endpoints
Model card Files Community
5
YC-Chen commited on
Commit
9f49a34
·
verified ·
1 Parent(s): 86172ea

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +13 -13
README.md CHANGED
@@ -54,19 +54,19 @@ and is comparable with Mistral-7B-Instruct-v0.1 on MMLU and MT-Bench in English.
54
 
55
  ## Chat Model Performance
56
 
57
- | Models | | TMMLU+ (ACC) | TMMLU+ (ACC) | DRCD (EM) | MT-Bench-tw (Score) | MMLU (ACC) | MMLU (ACC) | MT-Bench (Score) |
58
- |--------------------------------------------|--------|--------------|--------------|-----------|---------------------|------------|------------|------------------|
59
- | | |TC, Knowledge |TC, Knowledge |TC, Reasoning|TC, Chat |EN, Knowledge|EN, Knowledge|EN, Chat |
60
- | | | 0 shot | 5 shot | 3 shot | 0 shot | 0 shot | 5 shot | 0 shot |
61
- | gpt-3.5-turbo-1106 | | | | | 7.1 | | | 7.9 |
62
- | [Yi-34B-Chat](https://huggingface.co/01-ai/Yi-34B-Chat) | 34B | 54.87 | | | 6.9 | 71.04 | | 7.6 |
63
- | [Qwen-14B-Chat](https://huggingface.co/Qwen/Qwen-14B-Chat) | 14B | 48.41 | | | 6.4 | 64.91 | | 7.2 |
64
- | [Yi-6B-Chat](https://huggingface.co/01-ai/Yi-6B-Chat) | 6B | 44.79 | | | 5.0 | 59.45 | | 6.0 |
65
- | [**Breeze-7B-Instruct-v0.1**](https://huggingface.co/MediaTek-Research/Breeze-7B-Instruct-v0.1) | 7B | 41.61 | | | 5.7 | 63.26 | | 7.1 |
66
- | [**Breeze-7B-Instruct-64k-v0.1**](https://huggingface.co/MediaTek-Research/Breeze-7B-Instruct-64k-v0.1) | 7B | 40.99 | | | 5.5 | 63.68 | | 7.1 |
67
- | [Qwen-7B-Chat](https://huggingface.co/Qwen/Qwen-7B-Chat) | 7B | 40.02 | | | 5.4 | 55.94 | | 6.2 |
68
- | [Taiwan-LLM-13B-v2.0-chat](https://huggingface.co/yentinglin/Taiwan-LLM-13B-v2.0-chat) | 13B | 29.47 | | | 5.0 | 50.50 | | -* |
69
- | [Taiwan-LLM-7B-v2.1-chat](https://huggingface.co/yentinglin/Taiwan-LLM-7B-v2.1-chat) | 7B | 28.08 | | | 4.2 | 42.72 | | -* |
70
 
71
 
72
  \* Taiwan-LLM models responds to multi-turn questions (English) in Traditional Chinese.
 
54
 
55
  ## Chat Model Performance
56
 
57
+ | Models | | TMMLU+ (ACC) | TMMLU+ (ACC) | DRCD (EM) | Table (ACC) | MT-Bench-tw (Score) | MMLU (ACC) | MMLU (ACC) | MT-Bench (Score) |
58
+ |--------------------------------------------|--------|--------------|--------------|-----------|-------------|--------|------------|------------|------------------|
59
+ | | |TC, Knowledge |TC, Knowledge |TC, Reasoning|TC, Reasoning|TC, Chat |EN, Knowledge|EN, Knowledge|EN, Chat |
60
+ | | | 0 shot | 5 shot | 3 shot | 0 shot | 0 shot | 0 shot | 5 shot | 0 shot |
61
+ | gpt-3.5-turbo-1106 | | | | | | 7.1 | | | 7.9 |
62
+ | [Yi-34B-Chat](https://huggingface.co/01-ai/Yi-34B-Chat) | 34B | 54.87 | | | | 6.9 | 71.04 | | 7.6 |
63
+ | [Qwen-14B-Chat](https://huggingface.co/Qwen/Qwen-14B-Chat) | 14B | 48.41 | | | 41.67 | 6.4 | 64.91 | | 7.2 |
64
+ | [Yi-6B-Chat](https://huggingface.co/01-ai/Yi-6B-Chat) | 6B | 44.79 | | | 25.69 | 5.0 | 59.45 | | 6.0 |
65
+ | [**Breeze-7B-Instruct-v0.1**](https://huggingface.co/MediaTek-Research/Breeze-7B-Instruct-v0.1) | 7B | 41.61 | | | 45.83 | 5.7 | 63.26 | | 7.1 |
66
+ | [**Breeze-7B-Instruct-64k-v0.1**](https://huggingface.co/MediaTek-Research/Breeze-7B-Instruct-64k-v0.1) | 7B | 40.99 | | | 36.11 | 5.5 | 63.68 | | 7.1 |
67
+ | [Qwen-7B-Chat](https://huggingface.co/Qwen/Qwen-7B-Chat) | 7B | 40.02 | | | 33.33 | 5.4 | 55.94 | | 6.2 |
68
+ | [Taiwan-LLM-13B-v2.0-chat](https://huggingface.co/yentinglin/Taiwan-LLM-13B-v2.0-chat) | 13B | 29.47 | | | 23.61 | 5.0 | 50.50 | | -* |
69
+ | [Taiwan-LLM-7B-v2.1-chat](https://huggingface.co/yentinglin/Taiwan-LLM-7B-v2.1-chat) | 7B | 28.08 | | | 31.25 | 4.2 | 42.72 | | -* |
70
 
71
 
72
  \* Taiwan-LLM models responds to multi-turn questions (English) in Traditional Chinese.