ToastyPigeon commited on
Commit
bb97674
1 Parent(s): cd2cd22

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +56 -15
README.md CHANGED
@@ -3,42 +3,83 @@ base_model: []
3
  tags:
4
  - mergekit
5
  - merge
6
-
7
  ---
8
- # Psycet-V2
9
 
10
  This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit).
11
 
12
  ## Merge Details
13
  ### Merge Method
14
 
15
- This model was merged using the [linear](https://arxiv.org/abs/2203.05482) merge method.
 
 
 
 
 
 
16
 
17
  ### Models Merged
18
 
19
  The following models were included in the merge:
20
- * E:\ModelMerge\merges\Psycet-V2\Psycet
21
- * E:\ModelMerge\merges\Psycet-V2\Psycet-Reverse
22
 
23
  ### Configuration
24
 
25
  The following YAML configuration was used to produce this model:
26
 
27
  ```yaml
 
 
 
 
 
 
28
  dtype: float16
29
- merge_method: linear
 
30
  slices:
31
- - sources:
32
- - layer_range: [0, 62]
33
- model:
34
- model:
35
- path: E:\ModelMerge\merges\Psycet-V2\Psycet
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
36
  parameters:
37
  weight: 0.5
38
- - layer_range: [0, 62]
39
- model:
40
- model:
41
- path: E:\ModelMerge\merges\Psycet-V2\Psycet-Reverse
42
  parameters:
43
  weight: 0.5
 
 
44
  ```
 
3
  tags:
4
  - mergekit
5
  - merge
 
6
  ---
7
+ # Psyonic-Cetacean-20B-V2
8
 
9
  This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit).
10
 
11
  ## Merge Details
12
  ### Merge Method
13
 
14
+ This model was merged using the [linear](https://arxiv.org/abs/2203.05482) merge method on two stack-merged models.
15
+
16
+ The first is [jebcarter/psyonic-cetacean-20B](https://huggingface.co/jebcarter/psyonic-cetacean-20B)
17
+ (Orca first, reproduced so I didn't have to download that model on top of the components).
18
+ The second is the same recipe with the models reversed.
19
+
20
+ Since [jebcarter](https://huggingface.co/jebcarter) suggested this recipe, credit goes to him.
21
 
22
  ### Models Merged
23
 
24
  The following models were included in the merge:
25
+ * microsoft/Orca-2-13b
26
+ * KoboldAI/LLaMA2-13B-Psyfighter2
27
 
28
  ### Configuration
29
 
30
  The following YAML configuration was used to produce this model:
31
 
32
  ```yaml
33
+ models:
34
+ - model: microsoft/Orca-2-13b
35
+ parameters:
36
+ weight: 1.0
37
+ merge_method: task_arithmetic
38
+ base_model: TheBloke/Llama-2-13B-fp16
39
  dtype: float16
40
+ name: FlatOrca2
41
+ ---
42
  slices:
43
+ - sources:
44
+ - model: FlatOrca2
45
+ layer_range: [0, 16]
46
+ - sources:
47
+ - model: KoboldAI/LLaMA2-13B-Psyfighter2
48
+ layer_range: [8, 24]
49
+ - sources:
50
+ - model: FlatOrca2
51
+ layer_range: [17, 32]
52
+ - sources:
53
+ - model: KoboldAI/LLaMA2-13B-Psyfighter2
54
+ layer_range: [25, 40]
55
+ merge_method: passthrough
56
+ dtype: float16
57
+ name: Psycet
58
+ ---
59
+ slices:
60
+ - sources:
61
+ - model: KoboldAI/LLaMA2-13B-Psyfighter2
62
+ layer_range: [0, 16]
63
+ - sources:
64
+ - model: FlatOrca2
65
+ layer_range: [8, 24]
66
+ - sources:
67
+ - model: KoboldAI/LLaMA2-13B-Psyfighter2
68
+ layer_range: [17, 32]
69
+ - sources:
70
+ - model: FlatOrca2
71
+ layer_range: [25, 40]
72
+ merge_method: passthrough
73
+ dtype: float16
74
+ name: Psycet-Reverse
75
+ ---
76
+ models:
77
+ - model: Psycet
78
  parameters:
79
  weight: 0.5
80
+ - model: Psycet-Reverse
 
 
 
81
  parameters:
82
  weight: 0.5
83
+ merge_method: linear
84
+ dtype: float16
85
  ```