Dampfinchen xzuyn commited on
Commit
9757cd4
1 Parent(s): ea4905a

Update README.md & mergekit_config.yml (#5)

Browse files

- Update README.md & mergekit_config.yml (cd6c1b817e9bd7e82f5a287709984ed44b061e04)


Co-authored-by: xzuyn <xzuyn@users.noreply.huggingface.co>

Files changed (2) hide show
  1. README.md +14 -3
  2. mergekit_config.yml +37 -9
README.md CHANGED
@@ -1,6 +1,11 @@
1
  ---
2
  base_model:
3
  - Undi95/Meta-Llama-3-8B-Instruct-hf
 
 
 
 
 
4
  library_name: transformers
5
  tags:
6
  - mergekit
@@ -33,8 +38,12 @@ This model was merged using the [DARE](https://arxiv.org/abs/2311.03099) [TIES](
33
  ### Models Merged
34
 
35
  The following models were included in the merge:
36
- * llama-3-8B-ultra-instruct/InstructPart
37
- * llama-3-8B-ultra-instruct/RPPart
 
 
 
 
38
 
39
  ### Configuration
40
 
@@ -56,6 +65,7 @@ dtype: bfloat16
56
  base_model: Undi95/Meta-Llama-3-8B-hf
57
  name: RPPart
58
  ---
 
59
  models:
60
  - model: Weyaxi/Einstein-v6.1-Llama3-8B
61
  parameters:
@@ -71,6 +81,7 @@ base_model: Undi95/Meta-Llama-3-8B-hf
71
  dtype: bfloat16
72
  name: InstructPart
73
  ---
 
74
  models:
75
  - model: RPPart
76
  parameters:
@@ -106,4 +117,4 @@ Detailed results can be found [here](https://huggingface.co/datasets/open-llm-le
106
  |MMLU (5-Shot) |68.32|
107
  |TruthfulQA (0-shot) |52.80|
108
  |Winogrande (5-shot) |76.95|
109
- |GSM8k (5-shot) |70.36|
 
1
  ---
2
  base_model:
3
  - Undi95/Meta-Llama-3-8B-Instruct-hf
4
+ - Undi95/Llama-3-LewdPlay-8B-evo
5
+ - jondurbin/bagel-8b-v1.0
6
+ - Weyaxi/Einstein-v6.1-Llama3-8B
7
+ - VAGOsolutions/Llama-3-SauerkrautLM-8b-Instruct
8
+ - aaditya/OpenBioLLM-Llama3-8B
9
  library_name: transformers
10
  tags:
11
  - mergekit
 
38
  ### Models Merged
39
 
40
  The following models were included in the merge:
41
+ * Undi95/Meta-Llama-3-8B-Instruct-hf
42
+ * Undi95/Llama-3-LewdPlay-8B-evo
43
+ * jondurbin/bagel-8b-v1.0
44
+ * Weyaxi/Einstein-v6.1-Llama3-8B
45
+ * VAGOsolutions/Llama-3-SauerkrautLM-8b-Instruct
46
+ * aaditya/OpenBioLLM-Llama3-8B
47
 
48
  ### Configuration
49
 
 
65
  base_model: Undi95/Meta-Llama-3-8B-hf
66
  name: RPPart
67
  ---
68
+
69
  models:
70
  - model: Weyaxi/Einstein-v6.1-Llama3-8B
71
  parameters:
 
81
  dtype: bfloat16
82
  name: InstructPart
83
  ---
84
+
85
  models:
86
  - model: RPPart
87
  parameters:
 
117
  |MMLU (5-Shot) |68.32|
118
  |TruthfulQA (0-shot) |52.80|
119
  |Winogrande (5-shot) |76.95|
120
+ |GSM8k (5-shot) |70.36|
mergekit_config.yml CHANGED
@@ -1,15 +1,43 @@
1
- base_model: Undi95/Meta-Llama-3-8B-Instruct-hf
 
 
 
 
 
 
 
 
 
 
2
  dtype: bfloat16
 
 
 
 
 
 
 
 
 
 
 
 
 
 
3
  merge_method: dare_ties
4
- slices:
5
- - sources:
6
- - layer_range: [0, 32]
7
- model: llama-3-8B-ultra-instruct/RPPart
 
 
 
8
  parameters:
9
  weight: 0.39
10
- - layer_range: [0, 32]
11
- model: llama-3-8B-ultra-instruct/InstructPart
12
  parameters:
13
  weight: 0.26
14
- - layer_range: [0, 32]
15
- model: Undi95/Meta-Llama-3-8B-Instruct-hf
 
 
 
1
+ models:
2
+ - model: ChaoticNeutrals/Poppy_Porpoise-v0.7-L3-8B
3
+ parameters:
4
+ weight: 0.4
5
+ - model: Undi95/Llama-3-LewdPlay-8B-evo
6
+ parameters:
7
+ weight: 0.5
8
+ - model: jondurbin/bagel-8b-v1.0
9
+ parameters:
10
+ weight: 0.1
11
+ merge_method: dare_ties
12
  dtype: bfloat16
13
+ base_model: Undi95/Meta-Llama-3-8B-hf
14
+ name: RPPart
15
+ ---
16
+
17
+ models:
18
+ - model: Weyaxi/Einstein-v6.1-Llama3-8B
19
+ parameters:
20
+ weight: 0.6
21
+ - model: VAGOsolutions/Llama-3-SauerkrautLM-8b-Instruct
22
+ parameters:
23
+ weight: 0.3
24
+ - model: aaditya/OpenBioLLM-Llama3-8B
25
+ parameters:
26
+ weight: 0.1
27
  merge_method: dare_ties
28
+ base_model: Undi95/Meta-Llama-3-8B-hf
29
+ dtype: bfloat16
30
+ name: InstructPart
31
+ ---
32
+
33
+ models:
34
+ - model: RPPart
35
  parameters:
36
  weight: 0.39
37
+ - model: InstructPart
 
38
  parameters:
39
  weight: 0.26
40
+ merge_method: dare_ties
41
+ base_model: Undi95/Meta-Llama-3-8B-Instruct-hf
42
+ dtype: bfloat16
43
+ name: Llama-3-8B-Ultra-Instruct