icefog72 commited on
Commit
3cec680
1 Parent(s): 06aa230

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +49 -49
README.md CHANGED
@@ -1,49 +1,49 @@
1
- ---
2
- base_model: []
3
- library_name: transformers
4
- tags:
5
- - mergekit
6
- - merge
7
-
8
- ---
9
- # matMistral-7B
10
-
11
- This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit).
12
-
13
- ## Merge Details
14
- ### Merge Method
15
-
16
- This model was merged using the SLERP merge method.
17
-
18
- ### Models Merged
19
-
20
- The following models were included in the merge:
21
- * G:\FModels\Mistral-Instruct-dolphin-7B
22
- * G:\FModels\mathstral-Mistral-7B
23
-
24
- ### Configuration
25
-
26
- The following YAML configuration was used to produce this model:
27
-
28
- ```yaml
29
- slices:
30
- - sources:
31
- - model: G:\FModels\Mistral-Instruct-dolphin-7B
32
- layer_range: [0, 32]
33
- - model: G:\FModels\mathstral-Mistral-7B
34
- layer_range: [0, 32]
35
-
36
- merge_method: slerp
37
- base_model: G:\FModels\Mistral-Instruct-dolphin-7B
38
- parameters:
39
- t:
40
- - filter: self_attn
41
- value: [0, 0.5, 0.3, 0.7, 1]
42
- - filter: mlp
43
- value: [1, 0.5, 0.7, 0.3, 0]
44
- - value: 0.5 # fallback for rest of tensors
45
-
46
- dtype: bfloat16
47
-
48
-
49
- ```
 
1
+ ---
2
+ base_model: []
3
+ library_name: transformers
4
+ tags:
5
+ - mergekit
6
+ - merge
7
+
8
+ ---
9
+ # MInstDolphin29mathM-7B-v0.3
10
+
11
+ This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit).
12
+
13
+ ## Merge Details
14
+ ### Merge Method
15
+
16
+ This model was merged using the SLERP merge method.
17
+
18
+ ### Models Merged
19
+
20
+ The following models were included in the merge:
21
+ * G:\FModels\Mistral-Instruct-dolphin-7B
22
+ * G:\FModels\mathstral-Mistral-7B
23
+
24
+ ### Configuration
25
+
26
+ The following YAML configuration was used to produce this model:
27
+
28
+ ```yaml
29
+ slices:
30
+ - sources:
31
+ - model: G:\FModels\Mistral-Instruct-dolphin-7B
32
+ layer_range: [0, 32]
33
+ - model: G:\FModels\mathstral-Mistral-7B
34
+ layer_range: [0, 32]
35
+
36
+ merge_method: slerp
37
+ base_model: G:\FModels\Mistral-Instruct-dolphin-7B
38
+ parameters:
39
+ t:
40
+ - filter: self_attn
41
+ value: [0, 0.5, 0.3, 0.7, 1]
42
+ - filter: mlp
43
+ value: [1, 0.5, 0.7, 0.3, 0]
44
+ - value: 0.5 # fallback for rest of tensors
45
+
46
+ dtype: bfloat16
47
+
48
+
49
+ ```