Hugging Face's logo

umiyuki
/
Umievo-itr012-Gleipnir-7B

Text Generation
Transformers
Safetensors
Japanese
mistral
mergekit
Merge
text-generation-inference
Model card Files Community
umiyuki commited on
Commit
420a58e
·
verified ·
1 Parent(s): 7c81d2b

Upload README.md

Browse files
Files changed (1) hide show
  1. README.md +173 -0
README.md ADDED
@@ -0,0 +1,173 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ base_model: []
3
+ library_name: transformers
4
+ tags:
5
+ - mergekit
6
+ - merge
7
+
8
+ ---
9
+ # itr012_score3.93
10
+
11
+ This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit).
12
+
13
+ ## Merge Details
14
+ ### Merge Method
15
+
16
+ This model was merged using the [linear](https://arxiv.org/abs/2203.05482) merge method using /home/umiyuki/automerge/evol_merge_storage/input_models/Japanese-Chat-Umievo-itr004-7b_579282327 as a base.
17
+
18
+ ### Models Merged
19
+
20
+ The following models were included in the merge:
21
+ * /home/umiyuki/automerge/evol_merge_storage/input_models/Japanese-Starling-ChatV-7B_1737576410
22
+ * /home/umiyuki/automerge/evol_merge_storage/input_models/Ninja-v1-RP-expressive-v2_4102792561
23
+ * /home/umiyuki/automerge/evol_merge_storage/input_models/Vecteus-v1_4179808746
24
+
25
+ ### Configuration
26
+
27
+ The following YAML configuration was used to produce this model:
28
+
29
+ ```yaml
30
+ base_model: /home/umiyuki/automerge/evol_merge_storage/input_models/Japanese-Chat-Umievo-itr004-7b_579282327
31
+ dtype: bfloat16
32
+ merge_method: linear
33
+ parameters:
34
+ int8_mask: 1.0
35
+ normalize: 1.0
36
+ slices:
37
+ - sources:
38
+ - layer_range: [0, 4]
39
+ model: /home/umiyuki/automerge/evol_merge_storage/input_models/Japanese-Chat-Umievo-itr004-7b_579282327
40
+ parameters:
41
+ weight: 0.34953096474223655
42
+ - layer_range: [0, 4]
43
+ model: /home/umiyuki/automerge/evol_merge_storage/input_models/Vecteus-v1_4179808746
44
+ parameters:
45
+ weight: 0.4701212555597746
46
+ - layer_range: [0, 4]
47
+ model: /home/umiyuki/automerge/evol_merge_storage/input_models/Japanese-Starling-ChatV-7B_1737576410
48
+ parameters:
49
+ weight: 0.08162258723819021
50
+ - layer_range: [0, 4]
51
+ model: /home/umiyuki/automerge/evol_merge_storage/input_models/Ninja-v1-RP-expressive-v2_4102792561
52
+ parameters:
53
+ weight: 0.31015439852818116
54
+ - sources:
55
+ - layer_range: [4, 8]
56
+ model: /home/umiyuki/automerge/evol_merge_storage/input_models/Japanese-Chat-Umievo-itr004-7b_579282327
57
+ parameters:
58
+ weight: 0.11807412349683076
59
+ - layer_range: [4, 8]
60
+ model: /home/umiyuki/automerge/evol_merge_storage/input_models/Vecteus-v1_4179808746
61
+ parameters:
62
+ weight: -0.005684817244530085
63
+ - layer_range: [4, 8]
64
+ model: /home/umiyuki/automerge/evol_merge_storage/input_models/Japanese-Starling-ChatV-7B_1737576410
65
+ parameters:
66
+ weight: 0.2119283777941045
67
+ - layer_range: [4, 8]
68
+ model: /home/umiyuki/automerge/evol_merge_storage/input_models/Ninja-v1-RP-expressive-v2_4102792561
69
+ parameters:
70
+ weight: 1.1521124768396636
71
+ - sources:
72
+ - layer_range: [8, 12]
73
+ model: /home/umiyuki/automerge/evol_merge_storage/input_models/Japanese-Chat-Umievo-itr004-7b_579282327
74
+ parameters:
75
+ weight: 0.9244329405120573
76
+ - layer_range: [8, 12]
77
+ model: /home/umiyuki/automerge/evol_merge_storage/input_models/Vecteus-v1_4179808746
78
+ parameters:
79
+ weight: 0.7633842909616317
80
+ - layer_range: [8, 12]
81
+ model: /home/umiyuki/automerge/evol_merge_storage/input_models/Japanese-Starling-ChatV-7B_1737576410
82
+ parameters:
83
+ weight: 0.6952382990160072
84
+ - layer_range: [8, 12]
85
+ model: /home/umiyuki/automerge/evol_merge_storage/input_models/Ninja-v1-RP-expressive-v2_4102792561
86
+ parameters:
87
+ weight: 0.6873040403268571
88
+ - sources:
89
+ - layer_range: [12, 16]
90
+ model: /home/umiyuki/automerge/evol_merge_storage/input_models/Japanese-Chat-Umievo-itr004-7b_579282327
91
+ parameters:
92
+ weight: 0.4109625320908857
93
+ - layer_range: [12, 16]
94
+ model: /home/umiyuki/automerge/evol_merge_storage/input_models/Vecteus-v1_4179808746
95
+ parameters:
96
+ weight: 0.7090818691683626
97
+ - layer_range: [12, 16]
98
+ model: /home/umiyuki/automerge/evol_merge_storage/input_models/Japanese-Starling-ChatV-7B_1737576410
99
+ parameters:
100
+ weight: 0.42059423827890385
101
+ - layer_range: [12, 16]
102
+ model: /home/umiyuki/automerge/evol_merge_storage/input_models/Ninja-v1-RP-expressive-v2_4102792561
103
+ parameters:
104
+ weight: 0.5705186152354104
105
+ - sources:
106
+ - layer_range: [16, 20]
107
+ model: /home/umiyuki/automerge/evol_merge_storage/input_models/Japanese-Chat-Umievo-itr004-7b_579282327
108
+ parameters:
109
+ weight: 0.28507448659933315
110
+ - layer_range: [16, 20]
111
+ model: /home/umiyuki/automerge/evol_merge_storage/input_models/Vecteus-v1_4179808746
112
+ parameters:
113
+ weight: 0.4025223854083849
114
+ - layer_range: [16, 20]
115
+ model: /home/umiyuki/automerge/evol_merge_storage/input_models/Japanese-Starling-ChatV-7B_1737576410
116
+ parameters:
117
+ weight: 0.25885405316835886
118
+ - layer_range: [16, 20]
119
+ model: /home/umiyuki/automerge/evol_merge_storage/input_models/Ninja-v1-RP-expressive-v2_4102792561
120
+ parameters:
121
+ weight: 0.35540632690403373
122
+ - sources:
123
+ - layer_range: [20, 24]
124
+ model: /home/umiyuki/automerge/evol_merge_storage/input_models/Japanese-Chat-Umievo-itr004-7b_579282327
125
+ parameters:
126
+ weight: 0.018882795552694703
127
+ - layer_range: [20, 24]
128
+ model: /home/umiyuki/automerge/evol_merge_storage/input_models/Vecteus-v1_4179808746
129
+ parameters:
130
+ weight: 0.628847855051209
131
+ - layer_range: [20, 24]
132
+ model: /home/umiyuki/automerge/evol_merge_storage/input_models/Japanese-Starling-ChatV-7B_1737576410
133
+ parameters:
134
+ weight: 0.7038654876125734
135
+ - layer_range: [20, 24]
136
+ model: /home/umiyuki/automerge/evol_merge_storage/input_models/Ninja-v1-RP-expressive-v2_4102792561
137
+ parameters:
138
+ weight: 0.877501753107237
139
+ - sources:
140
+ - layer_range: [24, 28]
141
+ model: /home/umiyuki/automerge/evol_merge_storage/input_models/Japanese-Chat-Umievo-itr004-7b_579282327
142
+ parameters:
143
+ weight: 0.14008355431312197
144
+ - layer_range: [24, 28]
145
+ model: /home/umiyuki/automerge/evol_merge_storage/input_models/Vecteus-v1_4179808746
146
+ parameters:
147
+ weight: 1.0153826426873882
148
+ - layer_range: [24, 28]
149
+ model: /home/umiyuki/automerge/evol_merge_storage/input_models/Japanese-Starling-ChatV-7B_1737576410
150
+ parameters:
151
+ weight: 0.5586634927008272
152
+ - layer_range: [24, 28]
153
+ model: /home/umiyuki/automerge/evol_merge_storage/input_models/Ninja-v1-RP-expressive-v2_4102792561
154
+ parameters:
155
+ weight: 0.54455848971032
156
+ - sources:
157
+ - layer_range: [28, 32]
158
+ model: /home/umiyuki/automerge/evol_merge_storage/input_models/Japanese-Chat-Umievo-itr004-7b_579282327
159
+ parameters:
160
+ weight: 0.8188405381342685
161
+ - layer_range: [28, 32]
162
+ model: /home/umiyuki/automerge/evol_merge_storage/input_models/Vecteus-v1_4179808746
163
+ parameters:
164
+ weight: 0.5130358379308082
165
+ - layer_range: [28, 32]
166
+ model: /home/umiyuki/automerge/evol_merge_storage/input_models/Japanese-Starling-ChatV-7B_1737576410
167
+ parameters:
168
+ weight: 1.1132727871460124
169
+ - layer_range: [28, 32]
170
+ model: /home/umiyuki/automerge/evol_merge_storage/input_models/Ninja-v1-RP-expressive-v2_4102792561
171
+ parameters:
172
+ weight: 0.4471258297582539
173
+ ```