DavidAU commited on
Commit
7ecc41e
·
verified ·
1 Parent(s): d49bc80

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +80 -76
README.md CHANGED
@@ -1,76 +1,80 @@
1
- ---
2
- base_model: []
3
- library_name: transformers
4
- tags:
5
- - mergekit
6
- - merge
7
-
8
- ---
9
-
10
- <h2>L3-Stheno-v3.2-12.2B-Instruct - Float32</h2>
11
-
12
- This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit).
13
-
14
- ## Merge Details
15
- ### Merge Method
16
-
17
- This model was merged using the passthrough merge method.
18
-
19
- ### Models Merged
20
-
21
- The following models were included in the merge:
22
- * G:/7B/L3-8B-Stheno-v3.2
23
- * G:/7B/Meta-Llama-3-8B-Instruct
24
-
25
- ### Configuration
26
-
27
- The following YAML configuration was used to produce this model:
28
-
29
- ```yaml
30
- slices:
31
- - sources:
32
- - model: G:/7B/Meta-Llama-3-8B-Instruct
33
- layer_range: [0, 12]
34
- - sources:
35
- - model: G:/7B/L3-8B-Stheno-v3.2
36
- layer_range: [6, 19]
37
- parameters:
38
- scale:
39
- - filter: o_proj
40
- value: 1
41
- - filter: down_proj
42
- value: 1
43
- - value: 1
44
- - sources:
45
- - model: G:/7B/Meta-Llama-3-8B-Instruct
46
- layer_range: [12, 18]
47
- parameters:
48
- scale:
49
- - filter: o_proj
50
- value: .5
51
- - filter: down_proj
52
- value: .5
53
- - value: 1
54
- - sources:
55
- - model: G:/7B/Meta-Llama-3-8B-Instruct
56
- layer_range: [18, 25]
57
- parameters:
58
- scale:
59
- - filter: o_proj
60
- value: .75
61
- - filter: down_proj
62
- value: .75
63
- - value: 1
64
- - sources:
65
- - model: G:/7B/L3-8B-Stheno-v3.2
66
- layer_range: [19, 32]
67
- parameters:
68
- scale:
69
- - filter: o_proj
70
- value: 1
71
- - filter: down_proj
72
- value: 1
73
- - value: 1
74
- merge_method: passthrough
75
- dtype: float32
76
- ```
 
 
 
 
 
1
+ ---
2
+ base_model: []
3
+ library_name: transformers
4
+ tags:
5
+ - mergekit
6
+ - merge
7
+
8
+ ---
9
+
10
+ <h2>L3-Stheno-v3.2-12.2B-Instruct - Float32</h2>
11
+
12
+ For full model card, and GGUFs, please go to:
13
+
14
+ [ https://huggingface.co/DavidAU/L3-Stheno-v3.2-12.2B-INSTRUCT-ULTRA-F32-GGUF ]
15
+
16
+ This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit).
17
+
18
+ ## Merge Details
19
+ ### Merge Method
20
+
21
+ This model was merged using the passthrough merge method.
22
+
23
+ ### Models Merged
24
+
25
+ The following models were included in the merge:
26
+ * G:/7B/L3-8B-Stheno-v3.2
27
+ * G:/7B/Meta-Llama-3-8B-Instruct
28
+
29
+ ### Configuration
30
+
31
+ The following YAML configuration was used to produce this model:
32
+
33
+ ```yaml
34
+ slices:
35
+ - sources:
36
+ - model: G:/7B/Meta-Llama-3-8B-Instruct
37
+ layer_range: [0, 12]
38
+ - sources:
39
+ - model: G:/7B/L3-8B-Stheno-v3.2
40
+ layer_range: [6, 19]
41
+ parameters:
42
+ scale:
43
+ - filter: o_proj
44
+ value: 1
45
+ - filter: down_proj
46
+ value: 1
47
+ - value: 1
48
+ - sources:
49
+ - model: G:/7B/Meta-Llama-3-8B-Instruct
50
+ layer_range: [12, 18]
51
+ parameters:
52
+ scale:
53
+ - filter: o_proj
54
+ value: .5
55
+ - filter: down_proj
56
+ value: .5
57
+ - value: 1
58
+ - sources:
59
+ - model: G:/7B/Meta-Llama-3-8B-Instruct
60
+ layer_range: [18, 25]
61
+ parameters:
62
+ scale:
63
+ - filter: o_proj
64
+ value: .75
65
+ - filter: down_proj
66
+ value: .75
67
+ - value: 1
68
+ - sources:
69
+ - model: G:/7B/L3-8B-Stheno-v3.2
70
+ layer_range: [19, 32]
71
+ parameters:
72
+ scale:
73
+ - filter: o_proj
74
+ value: 1
75
+ - filter: down_proj
76
+ value: 1
77
+ - value: 1
78
+ merge_method: passthrough
79
+ dtype: float32
80
+ ```