martyn commited on
Commit
c08c204
1 Parent(s): b171151

add README

Browse files
Files changed (3) hide show
  1. .README.md.swp +0 -0
  2. .mergelist.yaml.swp +0 -0
  3. README.md +62 -0
.README.md.swp ADDED
Binary file (12.3 kB). View file
 
.mergelist.yaml.swp ADDED
Binary file (12.3 kB). View file
 
README.md ADDED
@@ -0,0 +1,62 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: apache-2.0
3
+ language:
4
+ - en
5
+ pipeline_tag: text-generation
6
+ inference: false
7
+ tags:
8
+ - dare
9
+ - super mario merge
10
+ - pytorch
11
+ - solar
12
+ - merge
13
+ ---
14
+
15
+ # solar megamerge 10.7b
16
+
17
+ The following models were merged with DARE using [https://github.com/martyn/safetensors-merge-supermario](https://github.com/martyn/safetensors-merge-supermario)
18
+
19
+ ## Mergelist
20
+
21
+ ```
22
+ models:
23
+ - model: upstage/SOLAR-10.7B-v1.0
24
+ - model: upstage/SOLAR-10.7B-Instruct-v1.0
25
+ parameters:
26
+ weight: 0.20
27
+ density: 0.8
28
+ - model: kyujinpy/SOLAR-Platypus-10.7B-v1
29
+ parameters:
30
+ weight: 0.19
31
+ density: 0.75
32
+ - model: We-Want-GPU/SOLAR-10.7B-orca-alpaca-gpt4-math
33
+ parameters:
34
+ weight: 0.18
35
+ density: 0.75
36
+ - model: maywell/Synatra-10.7B-v0.4
37
+ parameters:
38
+ weight: 0.18
39
+ density: 0.7
40
+ - model: kyujinpy/SOLAR-Platypus-10.7B-v2
41
+ parameters:
42
+ weight: 0.17
43
+ density: 0.7
44
+ - model: Sao10K/Frostwind-10.7B-v1
45
+ parameters:
46
+ weight: 0.16
47
+ density: 0.65
48
+ - model: rishiraj/meow
49
+ parameters:
50
+ weight: 0.15
51
+ density: 0.6
52
+ ```
53
+
54
+ ## Merge command
55
+
56
+ ```
57
+ python3 hf_merge.py mergelist.yaml solar-1
58
+ ```
59
+
60
+ ### Notes
61
+
62
+ * in the yaml: `p=weight` and `lambda=1/density`