Update README.md
Browse files
README.md
CHANGED
@@ -15,6 +15,49 @@ pipeline_tag: text-generation
|
|
15 |
**Training Corpus**
|
16 |
- [Intel/orca_dpo_pairs](https://huggingface.co/datasets/Intel/orca_dpo_pairs)
|
17 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
18 |
**Log**
|
19 |
- 2024.02.05: Initial version Upload
|
20 |
|
|
|
15 |
**Training Corpus**
|
16 |
- [Intel/orca_dpo_pairs](https://huggingface.co/datasets/Intel/orca_dpo_pairs)
|
17 |
|
18 |
+
**Explanation**
|
19 |
+
- Merge two base models using [mergekit](https://github.com/arcee-ai/mergekit) (slerp)
|
20 |
+
- Apply DPO to the merged model, just an adapter part is saved
|
21 |
+
- merge the adpater and the merged model
|
22 |
+
|
23 |
+
**Merge Script**
|
24 |
+
```
|
25 |
+
slices:
|
26 |
+
- sources:
|
27 |
+
- model: upstage/SOLAR-10.7B-Instruct-v1.0
|
28 |
+
layer_range: [0, 48]
|
29 |
+
- model: beomi/OPEN-SOLAR-KO-10.7B
|
30 |
+
layer_range: [0, 48]
|
31 |
+
merge_method: slerp
|
32 |
+
base_model: upstage/SOLAR-10.7B-Instruct-v1.0
|
33 |
+
parameters:
|
34 |
+
t:
|
35 |
+
- filter: self_attn
|
36 |
+
value: [0, 0.5, 0.3, 0.7, 1]
|
37 |
+
- filter: mlp
|
38 |
+
value: [1, 0.5, 0.7, 0.3, 0]
|
39 |
+
- value: 0.5 # fallback for rest of tensors
|
40 |
+
dtype: float16
|
41 |
+
```
|
42 |
+
|
43 |
+
**Score**
|
44 |
+
|Average|ARC|HellaSwag|MMLU|TruthfulQA|Winogrande|GSM8K|
|
45 |
+
|:---:|:---:|:---:|:---:|:---:|:---:|:---:|
|
46 |
+
|65.96|63.91|84.58|63.18|51.49|82|50.57|
|
47 |
+
|
48 |
+
**Usage**
|
49 |
+
```python
|
50 |
+
from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig
|
51 |
+
model = AutoModelForCausalLM.from_pretrained(
|
52 |
+
"dddsaty/SOLAR_Merge_Adapter_DPO_Orca",
|
53 |
+
low_cpu_mem_usage = True,
|
54 |
+
torch_dtype = torch.float16,
|
55 |
+
device_map = device_map,
|
56 |
+
)
|
57 |
+
|
58 |
+
tokenizer = AutoTokenizer.from_pretrained("dddsaty/SOLAR_Merge_Adapter_DPO_Orca")
|
59 |
+
```
|
60 |
+
|
61 |
**Log**
|
62 |
- 2024.02.05: Initial version Upload
|
63 |
|