Aratako commited on
Commit
ab64506
1 Parent(s): 5f44195

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +17 -0
README.md CHANGED
@@ -25,6 +25,23 @@ When using the model, please place these files in the same folder as the model.
25
  This model inherits the the [tongyi-qianwen license](https://huggingface.co/Qwen/Qwen1.5-72B-Chat/blob/main/LICENSE).
26
 
27
  ## Benchmark
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
28
 
29
  ## Merge config
30
  [mergekit_config.yml](./mergekit_moe_config.yml)
 
25
  This model inherits the the [tongyi-qianwen license](https://huggingface.co/Qwen/Qwen1.5-72B-Chat/blob/main/LICENSE).
26
 
27
  ## Benchmark
28
+ The benchmark score of the [mt-bench](https://github.com/lm-sys/FastChat/tree/main/fastchat/llm_judge) for this model and the two base models are as follows:
29
+
30
+ **1-turn, 4-bit quantization**
31
+ |Model|Size|Coding|Extraction|Humanities|Math|Reasoning|Roleplay|STEM|Writing|avg_score|
32
+ |---|---|---|---|---|---|---|---|---|---|---|
33
+ | Liberated-Qwen1.5-72B | 72B | **5.8** | 7.9 | 9.6 | 6.7 | 7.0 | **9.05** | 9.55 | **9.9** | 8.1875 |
34
+ | Qwen1.5-72B-Chat | 72B | 5.5 | **8.7** | 9.7 | **8.4** | 7.5 | 9.0 | 9.45 | 9.75 | **8.5000** |
35
+ | This model | 2x72B | 5.6 | 7.8 | **9.75** | 7.0 | **8.1** | 9.0 | **9.65** | 9.8 | 8.3375 |
36
+ ![mt-bench-1turn](./mt-bench-1turn.png)
37
+
38
+ **2-turn, 4-bit quantization**
39
+ |Model|Size|Coding|Extraction|Humanities|Math|Reasoning|Roleplay|STEM|Writing|avg_score|
40
+ |---|---|---|---|---|---|---|---|---|---|---|
41
+ | Liberated-Qwen1.5-72B | 72B | 3.9 | 8.2 | **10.0** | 5.7 | 5.5 | 8.4 | 8.7 | 8.6 | 7.3750 |
42
+ | Qwen1.5-72B-Chat | 72B | **5.2** | 8.8 | **10.0** | **6.1** | 6.7 | 9.0 | **9.8** | **9.5** | 8.1375 |
43
+ | This model | 2x72B | 5.0 | **9.5** | 9.9 | 5.6 | **8.1** | **9.3** | 9.6 | 9.2 | **8.2750** |
44
+ ![mt-bench-2turn](./mt-bench-2turn.png)
45
 
46
  ## Merge config
47
  [mergekit_config.yml](./mergekit_moe_config.yml)