MediaTek-Research
/

Breeze-7B-Instruct-v0_1

Text Generation

text-generation-inference

Inference Endpoints

Model card Files Files and versions Community

YC-Chen commited on Jan 11, 2024

Commit

fa611d6

·

verified ·

1 Parent(s): e0384e7

Update README.md

Files changed (1) hide show

README.md +9 -10

README.md CHANGED Viewed

@@ -90,18 +90,17 @@ and is comparable with Mistral-7B-Instruct-v0.1 on MMLU and MT-Bench in English.
 \* Taiwan-LLM models responds to multi-turn questions (English) in Traditional Chinese.
-| Category ACC of TMMLU+ (0 shot)                      | STEM         | Social Science | Humanities | Other      |
 |-----------------------------------------------------|--------------|----------------|------------|------------|
 | gpt-3.5-turbo-1106                                  |              |                |            |            |
-| Yi-34B-Chat                                   | 46.36        | 65.02          | 52.84      | 52.21      |
-| Qwen-14B-Chat                                  | 46.51        | 58.20          | 51.12      | 49.38      |
-| Yi-6B-Chat                                    | 26.28        | 33.48          | 29.48      | 27.62      |
-| Breeze-7B-Instruct-v0.1           | 37.45        | 48.35          | 40.26      | 40.44      |
-| Breeze-7B-Instruct-64k-v0.1           | 37.45        | 48.35          | 40.26      | 40.44      |
-| Qwen-7B-Chat                                   | 32.89        | 44.26          | 38.21      | 37.83      |
-| Taiwan-LLM-13B-v2.0-chat                 | 29.68        | 37.13          | 30.31      | 30.55      |
-| Taiwan-LLM-7B-v2.1-chat                  | 26.53        | 29.47          | 26.11      | 26.90      |
 ## Examples

 \* Taiwan-LLM models responds to multi-turn questions (English) in Traditional Chinese.
+| Category ACC of TMMLU+ (0 shot)                     | STEM         | Social Science | Humanities | Other      |
 |-----------------------------------------------------|--------------|----------------|------------|------------|
 | gpt-3.5-turbo-1106                                  |              |                |            |            |
+| Yi-34B-Chat                                         | 47.65        | 64.25          | 52.73      | 54.91      |
+| Qwen-14B-Chat                                       | 43.83        | 55.00          | 48.55      | 46.22      |
+| Yi-6B-Chat                                          | 37.80        | 51.74          | 45.36      | 44.25      |
+| Breeze-7B-Instruct-v0.1                             | 37.41        | 46.81          | 42.06      | 40.16      |
+| Breeze-7B-Instruct-64k-v0.1                         | 37.88        | 46.35          | 40.31      | 39.40      |
+| Qwen-7B-Chat                                        | 35.44        | 46.22          | 38.35      | 40.06      |
+| Taiwan-LLM-13B-v2.0-chat                            | 27.74        | 33.69          | 27.03      | 29.43      |
+| Taiwan-LLM-7B-v2.1-chat                             | 25.58        | 31.76          | 27.36      | 27.61      |
 ## Examples