Update README.md
Browse files
README.md
CHANGED
@@ -25,6 +25,23 @@ When using the model, please place these files in the same folder as the model.
|
|
25 |
This model inherits the the [tongyi-qianwen license](https://huggingface.co/Qwen/Qwen1.5-72B-Chat/blob/main/LICENSE).
|
26 |
|
27 |
## Benchmark
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
28 |
|
29 |
## Merge config
|
30 |
[mergekit_config.yml](./mergekit_moe_config.yml)
|
|
|
25 |
This model inherits the the [tongyi-qianwen license](https://huggingface.co/Qwen/Qwen1.5-72B-Chat/blob/main/LICENSE).
|
26 |
|
27 |
## Benchmark
|
28 |
+
The benchmark score of the [mt-bench](https://github.com/lm-sys/FastChat/tree/main/fastchat/llm_judge) for this model and the two base models are as follows:
|
29 |
+
|
30 |
+
**1-turn, 4-bit quantization**
|
31 |
+
|Model|Size|Coding|Extraction|Humanities|Math|Reasoning|Roleplay|STEM|Writing|avg_score|
|
32 |
+
|---|---|---|---|---|---|---|---|---|---|---|
|
33 |
+
| Liberated-Qwen1.5-72B | 72B | **5.8** | 7.9 | 9.6 | 6.7 | 7.0 | **9.05** | 9.55 | **9.9** | 8.1875 |
|
34 |
+
| Qwen1.5-72B-Chat | 72B | 5.5 | **8.7** | 9.7 | **8.4** | 7.5 | 9.0 | 9.45 | 9.75 | **8.5000** |
|
35 |
+
| This model | 2x72B | 5.6 | 7.8 | **9.75** | 7.0 | **8.1** | 9.0 | **9.65** | 9.8 | 8.3375 |
|
36 |
+
![mt-bench-1turn](./mt-bench-1turn.png)
|
37 |
+
|
38 |
+
**2-turn, 4-bit quantization**
|
39 |
+
|Model|Size|Coding|Extraction|Humanities|Math|Reasoning|Roleplay|STEM|Writing|avg_score|
|
40 |
+
|---|---|---|---|---|---|---|---|---|---|---|
|
41 |
+
| Liberated-Qwen1.5-72B | 72B | 3.9 | 8.2 | **10.0** | 5.7 | 5.5 | 8.4 | 8.7 | 8.6 | 7.3750 |
|
42 |
+
| Qwen1.5-72B-Chat | 72B | **5.2** | 8.8 | **10.0** | **6.1** | 6.7 | 9.0 | **9.8** | **9.5** | 8.1375 |
|
43 |
+
| This model | 2x72B | 5.0 | **9.5** | 9.9 | 5.6 | **8.1** | **9.3** | 9.6 | 9.2 | **8.2750** |
|
44 |
+
![mt-bench-2turn](./mt-bench-2turn.png)
|
45 |
|
46 |
## Merge config
|
47 |
[mergekit_config.yml](./mergekit_moe_config.yml)
|