File size: 20,781 Bytes
3d4d706
 
 
f2f302c
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
e48003f
f2f302c
 
e48003f
 
 
f2f302c
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
---
license: apache-2.0
---
# FuseChat-7B-VaRM

## Description
This repo contains GGUF format model files for FuseChat-7B-VaRM.

## Files Provided
| Name                          |  Quant  | Bits | File Size |            Remark                |
| ----------------------------- | ------- | ---- | --------- | -------------------------------- |
| fusechat-7b-varm.IQ3_XXS.gguf | IQ3_XXS |  3   |  3.02 GB  | 3.06 bpw quantization            |
| fusechat-7b-varm.IQ3_S.gguf   | IQ3_S   |  3   |  3.18 GB  | 3.44 bpw quantization            |
| fusechat-7b-varm.IQ3_M.gguf   | IQ3_M   |  3   |  3.28 GB  | 3.66 bpw quantization mix        |
| fusechat-7b-varm.Q4_0.gguf    | Q4_0    |  4   |  4.11 GB  | 3.56G, +0.2166 ppl               |
| fusechat-7b-varm.IQ4_NL.gguf  | IQ4_NL  |  4   |  4.16 GB  | 4.25 bpw non-linear quantization |
| fusechat-7b-varm.Q4_K_M.gguf  | Q4_K_M  |  4   |  4.37 GB  | 3.80G, +0.0532 ppl               |
| fusechat-7b-varm.Q5_K_M.gguf  | Q5_K_M  |  5   |  5.13 GB  | 4.45G, +0.0122 ppl               |
| fusechat-7b-varm.Q6_K.gguf    | Q6_K    |  6   |  5.94 GB  | 5.15G, +0.0008 ppl               |
| fusechat-7b-varm.Q8_0.gguf    | Q8_0    |  8   |  7.70 GB  | 6.70G, +0.0004 ppl               |

## Parameters
| path                    | type    | architecture       | rope_theta | sliding_win | max_pos_embed |
| ----------------------- | ------- | ------------------ | ---------- | ----------- | ------------- |
| FuseAI/FuseChat-7B-VaRM | mistral | MistralForCausalLM | 10000.0    | 4096        | 8192          |

## Benchmarks
![](https://i.ibb.co/Z2WJqbT/Fuse-Chat-7-B-Va-RM.png)

## Specific Purpose Notes

This model has high MT-Bench score, comparable to GPT-3.5-Turbo. See original model card below.

# Original Model Card

---
license: apache-2.0
language:
- en
pipeline_tag: text-generation
tags:
- mistral
- mixtral
- solar
- model-fusion
- fusechat
library_name: transformers
---
<p align="center" width="100%">
</p>

<div id="top" align="center">

<p style="font-size: 32px; font-weight: bold;">FuseChat: Knowledge Fusion of Chat Models</p>


<h4> |<a href="https://arxiv.org/abs/2402.16107"> πŸ“‘ Paper </a> |
<a href="https://huggingface.co/FuseAI"> πŸ€— Huggingface Repo </a> |
<a href="https://github.com/fanqiwan/FuseLLM"> 🐱 Github Repo </a> |
</h4>

<!-- **Authors:** -->

_**Fanqi Wan, Ziyi Yang, Longguang Zhong, Xiaojun Quan, Xinting Huang, Wei Bi**_


<!-- **Affiliations:** -->


_Sun Yat-sen University_

<p align="center">
    <img src="./assets/fig_0.png" width="70%"> <br>
</p>

| Proprietary Models                                                    | #Params | MT-Bench | Open Source Models                                                    | #Params | MT-Bench |
|-----------------------------------------------------------------------|---------|----------|-----------------------------------------------------------------------|---------|----------|
| GPT-4-1106-preview                                                    | -       | 9.32     | Qwen1.5-72B-Chat                                                      | 72B     | 8.61     |
| GPT-4-0613                                                            | -       | 9.18     | Nous-Hermes-2-Mixtral-8x7B-DPO                                        | 8x7B    | 8.33     |
| GPT-4-0314                                                            | -       | 8.96     | Mixtral-8x7B-Instruct-v0.1                                            | 8x7B    | 8.30     |
| Mistral Medium                                                        | -       | 8.61     | πŸ€— [FuseChat-7B-VaRM](https://huggingface.co/FuseAI/FuseChat-7B-VaRM) | 7B      | 8.22     |
| GPT-3.5-Turbo-0613                                                    | -       | 8.39     | Starling-LM-7B-alpha                                                  | 7B      | 8.09     |
| GPT-3.5-Turbo-1106                                                    | -       | 8.32     | Tulu-2-DPO-70B                                                        | 70B     | 7.89     |
| πŸ€— [FuseChat-7B-VaRM](https://huggingface.co/FuseAI/FuseChat-7B-VaRM) | 7B      | 8.22     | OpenChat-3.5                                                          | 7B      | 7.81     |
| Claude-2.1                                                            | -       | 8.18     | OpenChat-3.5-0106                                                     | 7B      | 7.80     |
| Claude-2.0                                                            | -       | 8.06     | WizardLM-70B-v1.0                                                     | 70B     | 7.71     |
| GPT-3.5-Turbo-0314                                                    | -       | 7.94     | Yi-34B-Chat                                                           | 34B     | 7.67     |
| Claude-1                                                              | -       | 7.90     | Nous-Hermes-2-SOLAR-10.7B                                             | 10.7B   | 7.66     |


</div>


## News
- **Feb 26, 2024:** πŸ”₯ We release [FuseChat-7B-VaRM](https://huggingface.co/FuseAI/FuseChat-7B-VaRM), which is the fusion of three prominent chat LLMs with diverse architectures and scales, namely [NH2-Mixtral-8x7B](https://huggingface.co/NousResearch/Nous-Hermes-2-Mixtral-8x7B-DPO), [NH2-Solar-10.7B](https://huggingface.co/NousResearch/Nous-Hermes-2-SOLAR-10.7B), and [OpenChat-3.5-7B](https://huggingface.co/openchat/openchat_3.5). FuseChat-7B-VaRM achieves an average performance of **8.22** on MT-Bench, outperforming various powerful chat LLMs at 7B and 34B scales like [Starling-7B](https://huggingface.co/berkeley-nest/Starling-LM-7B-alpha) and [Yi-34B-Chat](https://huggingface.co/01-ai/Yi-34B-Chat), even surpassing [GPT-3.5 (March)](https://platform.openai.com/docs/models/gpt-3-5-turbo), [Claude-2.1](https://www.anthropic.com/news/claude-2-1), and approaching [Mixtral-8x7B-Instruct](https://huggingface.co/mistralai/Mixtral-8x7B-Instruct-v0.1). 


## Contents

- [Overview](#overview)
- [Model Release](#model-release)
- [Quick Start](#quick-start)
- [Data Construction](#data-construction)
- [Pairwise Knowledge Fusion](#pairwise-knowledge-fusion)
- [Model Merging](#model-merging)
- [Evaluation](#evaluation)
- [Citation](#citation)

## Overview
 
In this work, we propose an extended framework of FuseLLM to integrate the collective knowledge and individual strengths of multiple structure and scale-varied chat LLMs into a more powerful chat LLM, resulting in FuseChat. FuseChat adopts a fuse-then-merge strategy with two main stages. Firstly, it undertakes pairwise knowledge fusion for source LLMs to derive multiple target LLMs of identical structure and size via lightweight fine-tuning. Then, these target LLMs are merged within the parameter space, wherein we propose a novel method VaRM for determining the merging weights based on the variation ratio of parameter matrices before and after fine-tuning. 


Moreover, we argue that the concept of knowledge fusion adopted by both FuseChat and FuseLLM shares a fundamentally similar purpose with other related topics, such as the recently popular topic of mixture of experts (MoEs), because they all aim to leverage the strengths of multiple models (experts). However, while MoEs require loading multiple experts during inference, which has higher memory requirements, knowledge fusion supports the integration of multiple LLMs with diverse architectures into a single LLM without any additional memory requirement, making it more memory-efficient. 

<p align="center">
    <img src="./assets/fig_1.png" width="95%"> <br>
</p>


## Model Release

We release [FuseChat-7B-VaRM](https://huggingface.co/FuseAI/FuseChat-7B-VaRM), which is the fusion of three prominent chat LLMs with diverse architectures and scales, namely [NH2-Mixtral-8x7B](https://huggingface.co/NousResearch/Nous-Hermes-2-Mixtral-8x7B-DPO), [NH2-Solar-10.7B](https://huggingface.co/NousResearch/Nous-Hermes-2-SOLAR-10.7B), and [OpenChat-3.5-7B](https://huggingface.co/openchat/openchat_3.5). FuseChat-7B-VaRM achieves an average performance of **8.22** on MT-Bench, outperforming various powerful chat LLMs at 7B and 34B scales like [Starling-7B](https://huggingface.co/berkeley-nest/Starling-LM-7B-alpha) and [Yi-34B-Chat](https://huggingface.co/01-ai/Yi-34B-Chat), even surpassing [GPT-3.5 (March)](https://platform.openai.com/docs/models/gpt-3-5-turbo), [Claude-2.1](https://www.anthropic.com/news/claude-2-1), and approaching [Mixtral-8x7B-Instruct](https://huggingface.co/mistralai/Mixtral-8x7B-Instruct-v0.1).

To support a plug-and-play fusion of new source LLM, we release our target LLMs: [OpenChat-3.5-7B-Solar](https://huggingface.co/FuseAI/OpenChat-3.5-7B-Solar) and [OpenChat-3.5-7B-Mixtral](https://huggingface.co/FuseAI/OpenChat-3.5-7B-Mixtral), which are obtained from pair-wise knowledge fusion. Integrating a new source LLM at any scale requires only obtaining a target LLM from the new source LLM and merging it with the existing target LLMs.

We also release FuseChat with other merging methods: [FuseChat-7B-SLERP](https://huggingface.co/FuseAI/FuseChat-7B-SLERP) and [FuseChat-7B-TA](https://huggingface.co/FuseAI/FuseChat-7B-TA), which achieves an average performance of **8.19** and **8.20** on MT-Bench respectively.

Here are the evaluation results.

<p align="center">
    <img src="./assets/tab_1.png" width="95%"> <br>
</p>

## Quick Start

### Setup

We use `python 3.11` in this project.

Then, we have to install all the libraries listed in `requirements.txt`.

```bash
pip install -r requirements.txt
```

### Usage

Here's how you can run the model using the πŸ€— Transformers:

```python
import transformers
tokenizer = transformers.AutoTokenizer.from_pretrained("FuseAI/FuseChat-7B-VaRM")
# Single-turn
tokens = tokenizer("GPT4 Correct User: Hello<|end_of_turn|>GPT4 Correct Assistant:").input_ids
assert tokens == [1, 420, 6316, 28781, 3198, 3123, 1247, 28747, 22557, 32000, 420, 6316, 28781, 3198, 3123, 21631, 28747]
# Multi-turn
tokens = tokenizer("GPT4 Correct User: Hello<|end_of_turn|>GPT4 Correct Assistant: Hi<|end_of_turn|>GPT4 Correct User: How are you today?<|end_of_turn|>GPT4 Correct Assistant:").input_ids
assert tokens == [1, 420, 6316, 28781, 3198, 3123, 1247, 28747, 22557, 32000, 420, 6316, 28781, 3198, 3123, 21631, 28747, 15359, 32000, 420, 6316, 28781, 3198, 3123, 1247, 28747, 1602, 460, 368, 3154, 28804, 32000, 420, 6316, 28781, 3198, 3123, 21631, 28747]
```

The GPT4 template is also available as the integrated `tokenizer.chat_template`, which can be used instead of manually specifying the template:

```python
messages = [
    {"role": "user", "content": "Hello"},
    {"role": "assistant", "content": "Hi"},
    {"role": "user", "content": "How are you today?"}
]
tokens = tokenizer.apply_chat_template(messages, add_generation_prompt=True)
assert tokens == [1, 420, 6316, 28781, 3198, 3123, 1247, 28747, 22557, 32000, 420, 6316, 28781, 3198, 3123, 21631, 28747, 15359, 32000, 420, 6316, 28781, 3198, 3123, 1247, 28747, 1602, 460, 368, 3154, 28804, 32000, 420, 6316, 28781, 3198, 3123, 21631, 28747]
```

## Data Construction

We curated a comprehensive training dataset, [FuseChat-Mixture](https://huggingface.co/datasets/FuseAI/FuseChat-Mixture), from various sources. This dataset covers different styles and capabilities, featuring both human-written and model-generated, and spanning general instruction-following and specific skills. 

Here we show the scripts to obtain representations from multiple source LLMs for model fusion.

1. Get representations for each source LLM

```bash
# We split the dataset into 4 splits, then process each split on one or multiple GPU.

# OpenChat-3.5-7B
export CUDA_VISIBLE_DEVICES=0
for i in {0..3}; do
python /train/get_data_representation.py \
  --model_name_or_path "openchat/openchat_3.5" \
  --data_path "/data/fusechat_v1_clean_split_2048_filter_wrong.json" \
  --dataset_save_dir "<${i}_4_path_to_openchat_representation>" \
  --tknz_dataset_path "<${i}_4_path_to_openchat_tknz>" \
  --cache_dir "/.cache/huggingface/datasets" \
  --model_max_length 2048 \
  --load_in_half bf16 \
  --batch_size 32 \
  --top_k_logits 10 \
  --save_per_token_metric \
  --no_assert \
  --conv_temp "openchat" \
  --flash_attn_transformers \
  --mask_instruction \
  --dataset_split_num 4 \
  --dataset_index ${i}
done 

# NH2-Mixtral-8x7B
export CUDA_VISIBLE_DEVICES=0,1,2
for i in {0..3}; do
python /train/get_data_representation.py \
  --model_name_or_path "NousResearch/Nous-Hermes-2-Mixtral-8x7B-DPO" \
  --data_path "/data/fusechat_v1_clean_split_2048_filter_wrong.json" \
  --dataset_save_dir "<${i}_4_path_to_mixtral_representation>" \
  --tknz_dataset_path "<${i}_4_path_to_mixtral_tknz>" \
  --cache_dir "/.cache/huggingface/datasets" \
  --model_max_length 2048 \
  --load_in_half bf16 \
  --batch_size 4 \
  --top_k_logits 10 \
  --save_per_token_metric \
  --no_assert \
  --conv_temp "openchat" \
  --flash_attn_transformers \
  --mask_instruction \
  --device_map "auto" \
  --dataset_split_num 4 \
  --dataset_index ${i}
done 

# NH2-Solar-10.7B
export CUDA_VISIBLE_DEVICES=0
for i in {0..3}; do
python /train/get_data_representation.py \
  --model_name_or_path "NousResearch/Nous-Hermes-2-SOLAR-10.7B" \
  --data_path "/data/fusechat_v1_clean_split_2048_filter_wrong.json" \
  --dataset_save_dir "<${i}_4_path_to_solar_representation>" \
  --tknz_dataset_path "<${i}_4_path_to_solar_tknz>" \
  --cache_dir "/.cache/huggingface/datasets" \
  --model_max_length 2048 \
  --load_in_half bf16 \
  --batch_size 8 \
  --top_k_logits 10 \
  --save_per_token_metric \
  --no_assert \
  --conv_temp "openchat" \
  --flash_attn_transformers \
  --mask_instruction \
  --dataset_split_num 4 \
  --dataset_index ${i}
done 
```

2. Align representations from different source LLMs

```bash
# Since the tokenizers and vocabularies of these source LLMs are identical, we do not align.

# OpenChat-3.5-7B <-> NH2-Mixtral-8x7B
for i in {0..3}; do
python /train/replace_model.py \
  --dataset_dir "<${i}_4_path_to_openchat_representation>" \
  --replace_dataset_dir "<${i}_4_path_to_mixtral_representation>" \
  --dataset_save_dir "<${i}_4_path_to_openchat_mixtral_representation>" \
  --preprocessing_num_workers 64 \
  --batch_size 1000 \
  --replace_model model_0
done 

# OpenChat-3.5-7B <-> NH2-Solar-10.7B
for i in {0..3}; do
python /train/replace_model.py \
  --dataset_dir "<${i}_4_path_to_openchat_mixtral_representation>" \
  --replace_dataset_dir "<${i}_4_path_to_solar_representation>" \
  --dataset_save_dir "<${i}_4_path_to_openchat_mixtral_solar_representation>" \
  --preprocessing_num_workers 64 \
  --batch_size 1000 \
  --replace_model model_1
done
```

3. Filter instances with NaN loss in the dataset

```bash
for i in {0..3}; do
python /train/filter_nan.py \
  --input_data_dir "<${i}_4_path_to_openchat_mixtral_solar_representation>" \
  --output_data_dir "<${i}_4_path_to_openchat_mixtral_solar_representation_fnan>"
done
```

The final processed data is at `<${i}_4_path_to_openchat_mixtral_solar_representation_fnan>`.

## Pairwise Knowledge Fusion

We show the scripts for pairwise knowledge fusion.

```bash
# OpenChat-3.5-7B <-> NH2-Mixtral-8x7B
export CUDA_VISIBLE_DEVICES=0,1,2,3,4,5,6,7
torchrun --nproc_per_node=8 --master_port=20001 /train/train.py \
  --model_name_or_path "openchat/openchat_3.5" \
  --data_path "<0_4_path_to_openchat_mixtral_solar_representation_fnan>,<1_4_path_to_openchat_mixtral_solar_representation_fnan>,<2_4_path_to_openchat_mixtral_solar_representation_fnan>,<3_4_path_to_openchat_mixtral_solar_representation_fnan>" \
  --bf16 True \
  --output_dir "<path_to_save_openchat_mixtral_ckpt>" \
  --num_train_epochs 3 \
  --per_device_train_batch_size 4 \
  --per_device_eval_batch_size 4 \
  --gradient_accumulation_steps 4 \
  --evaluation_strategy "no" \
  --save_strategy "epoch" \
  --save_steps 10000 \
  --save_total_limit 5 \
  --learning_rate 5e-6 \
  --weight_decay 0. \
  --warmup_ratio 0.03 \
  --lr_scheduler_type "cosine" \
  --logging_steps 1 \
  --fsdp "full_shard auto_wrap" \
  --fsdp_transformer_layer_cls_to_wrap 'MistralDecoderLayer' \
  --tf32 True \
  --model_max_length 2048 \
  --gradient_checkpointing True \
  --conv_temp "openchat" \
  --lazy_preprocess True \
  --flash_attn_transformers True \
  --do_train \
  --do_distill \
  --distill_with_ref_model True \
  --distill_with_aligned_model_0 True \
  --distill_with_aligned_model_1 False \
  --distill_loss_type "ce" \
  --distill_teacher_temperature 1.0 \
  --lm_loss_weight 0.9 \
  --distill_greater_as_gt True \
  --distill_greater_as_gt_type hard \
  --dataloader_num_workers 8 \
  --remove_unused_columns False

# OpenChat-3.5-7B <-> NH2-Solar-10.7B
export CUDA_VISIBLE_DEVICES=0,1,2,3,4,5,6,7
torchrun --nproc_per_node=8 --master_port=20001 /train/train.py \
  --model_name_or_path "openchat/openchat_3.5" \
  --data_path "<0_4_path_to_openchat_mixtral_solar_representation_fnan>,<1_4_path_to_openchat_mixtral_solar_representation_fnan>,<2_4_path_to_openchat_mixtral_solar_representation_fnan>,<3_4_path_to_openchat_mixtral_solar_representation_fnan>" \
  --bf16 True \
  --output_dir "<path_to_save_openchat_solar_ckpt>" \
  --num_train_epochs 3 \
  --per_device_train_batch_size 4 \
  --per_device_eval_batch_size 4 \
  --gradient_accumulation_steps 4 \
  --evaluation_strategy "no" \
  --save_strategy "epoch" \
  --save_steps 10000 \
  --save_total_limit 5 \
  --learning_rate 5e-6 \
  --weight_decay 0. \
  --warmup_ratio 0.03 \
  --lr_scheduler_type "cosine" \
  --logging_steps 1 \
  --fsdp "full_shard auto_wrap" \
  --fsdp_transformer_layer_cls_to_wrap 'MistralDecoderLayer' \
  --tf32 True \
  --model_max_length 2048 \
  --gradient_checkpointing True \
  --conv_temp "openchat" \
  --lazy_preprocess True \
  --flash_attn_transformers True \
  --do_train \
  --do_distill \
  --distill_with_ref_model True \
  --distill_with_aligned_model_0 False \
  --distill_with_aligned_model_1 True \
  --distill_loss_type "ce" \
  --distill_teacher_temperature 1.0 \
  --lm_loss_weight 0.9 \
  --distill_greater_as_gt True \
  --distill_greater_as_gt_type hard \
  --dataloader_num_workers 8 \
  --remove_unused_columns False
```

## Model Merging

We show the scripts to obtain the final FuseChat using different merging methods.

```bash
# For "slerp", "ta", "ties", and "dare" methods (Please install "mergekit")
export CUDA_VISIBLE_DEVICES=0
mergekit-yaml merge/mergekit_configs/fusechat-slerp.yml "<path_to_save_fusechat_7b_slerp>"
mergekit-yaml merge/mergekit_configs/fusechat-ta.yml "<path_to_save_fusechat_7b_ta>"
mergekit-yaml merge/mergekit_configs/fusechat-ties.yml "<path_to_save_fusechat_7b_ties>"
mergekit-yaml merge/mergekit_configs/fusechat-dare.yml "<path_to_save_fusechat_7b_dare>"

# For "linear" method 
python merge/VaRM/merge.py \
  --merged_model_names "FuseAI/OpenChat-3.5-7B-Mixtral,FuseAI/OpenChat-3.5-7B-Solar" \
  --merged_model_save_dir "<path_to_save_fusechat_7b_linear>" \
  --merge_method "linear" \
  --linear_weights "1,2"

# For our "varm" method
python merge/VaRM/analysis.py \
  --model1_path "FuseAI/OpenChat-3.5-7B-Mixtral" \
  --model2_path "FuseAI/OpenChat-3.5-7B-Solar" \
  --save_path "<path_to_save_analysis_result>/analysis.json" \
  --merge_type "square"

python merge/VaRM/merge.py \
  --merged_model_names "FuseAI/OpenChat-3.5-7B-Mixtral,FuseAI/OpenChat-3.5-7B-Solar" \
  --analysis_result "<path_to_save_analysis_result>/analysis.json" \
  --merged_model_save_dir "<path_to_save_fusechat_7b_varm>" \
  --merge_method "avg_param" \
  --merge_type "square"
```

## Evaluation

We evaluate FuseChat on MT-Bench, which comprises 80 multi-turn dialogues spanning writing, roleplay, reasoning, math, coding, stem, and humanities domains. Please download the [official code](https://github.com/lm-sys/FastChat/tree/main/fastchat/llm_judge) and follow the guidelines for evaluation. We provide the scripts for our evaluation. 

```bash
# Step 1. Generate model answers to MT-bench questions
export CUDA_VISIBLE_DEVICES=0,1
python gen_model_answer.py \
  --model-path "FuseAI/FuseChat-7B-VaRM" \
  --model-id "openchat_3.5_fusechat_7b_varm" \
  --num-gpus-per-model 1 \
  --num-gpus-total 2

# Step 2. Generate GPT-4 judgments
export OPENAI_API_KEY=XXXXXX  # set the OpenAI API key
python gen_judgment.py \
  --parallel 2

# Step 3. Show MT-bench scores
python show_result.py
```

## Citation

If you find this work is relevant with your research or applications, please feel free to cite our work!
```
@article{wan2024fusechat,
  title={FuseChat: Knowledge Fusion of Chat Models},
  author={Fanqi Wan and Ziyi Yang and Longguang Zhong and Xiaojun Quan and Xinting Huang and Wei Bi},
  journal={arXiv preprint arXiv:2402.16107},
  year={2024}
}
```