Masterjp123 commited on
Commit
f17a801
1 Parent(s): 54869dc

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +47 -1
README.md CHANGED
@@ -15,6 +15,9 @@ tags:
15
  This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit).
16
 
17
  ## Merge Details
 
 
 
18
  ### Merge Method
19
 
20
  This model was merged using the [TIES](https://arxiv.org/abs/2306.01708) merge method using [TheBloke/Llama-2-13B-fp16](https://huggingface.co/TheBloke/Llama-2-13B-fp16) as a base.
@@ -23,7 +26,9 @@ This model was merged using the [TIES](https://arxiv.org/abs/2306.01708) merge m
23
 
24
  The following models were included in the merge:
25
  * [Masterjp123/SnowyRP-FinalV1-L2-13B](https://huggingface.co/Masterjp123/SnowyRP-FinalV1-L2-13B)
26
- * [Masterjp123/Snowyrp-V2B-P1](https://huggingface.co/Masterjp123/Snowyrp-V2B-P1)
 
 
27
  * [sauce1337/BerrySauce-L2-13b](https://huggingface.co/sauce1337/BerrySauce-L2-13b)
28
 
29
  ### Configuration
@@ -70,3 +75,44 @@ slices:
70
  model:
71
  path: TheBloke/Llama-2-13B-fp16
72
  ```
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
15
  This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit).
16
 
17
  ## Merge Details
18
+
19
+ Made as a test model, not sure about quality, probably will not make any quants unless someone finds out it's good and asks.
20
+
21
  ### Merge Method
22
 
23
  This model was merged using the [TIES](https://arxiv.org/abs/2306.01708) merge method using [TheBloke/Llama-2-13B-fp16](https://huggingface.co/TheBloke/Llama-2-13B-fp16) as a base.
 
26
 
27
  The following models were included in the merge:
28
  * [Masterjp123/SnowyRP-FinalV1-L2-13B](https://huggingface.co/Masterjp123/SnowyRP-FinalV1-L2-13B)
29
+ * [posicube/Llama2-chat-AYB-13B](https://huggingface.co/posicube/Llama2-chat-AYB-13B)
30
+ * [Sao10K/Stheno-1.8-L2-13B](https://huggingface.co/Sao10K/Stheno-1.8-L2-13B)
31
+ * [ValiantLabs/ShiningValiantXS](https://huggingface.co/ValiantLabs/ShiningValiantXS)
32
  * [sauce1337/BerrySauce-L2-13b](https://huggingface.co/sauce1337/BerrySauce-L2-13b)
33
 
34
  ### Configuration
 
75
  model:
76
  path: TheBloke/Llama-2-13B-fp16
77
  ```
78
+ for Masterjp123/Snowyrp-V2B-P1
79
+ ```yaml
80
+ base_model:
81
+ model:
82
+ path: TheBloke/Llama-2-13B-fp16
83
+ dtype: bfloat16
84
+ merge_method: ties
85
+ parameters:
86
+ int8_mask: 1.0
87
+ normalize: 1.0
88
+ slices:
89
+ - sources:
90
+ - layer_range: [0, 40]
91
+ model:
92
+ model:
93
+ path: Sao10K/Stheno-1.8-L2-13B
94
+ parameters:
95
+ density: [1.0, 0.7, 0.1]
96
+ weight: 1.0
97
+ - layer_range: [0, 40]
98
+ model:
99
+ model:
100
+ path: ValiantLabs/ShiningValiantXS
101
+ parameters:
102
+ density: 0.5
103
+ weight: [0.0, 0.3, 0.7, 1.0]
104
+ - layer_range: [0, 40]
105
+ model:
106
+ model:
107
+ path: posicube/Llama2-chat-AYB-13B
108
+ parameters:
109
+ density: 0.33
110
+ weight:
111
+ - filter: mlp
112
+ value: 0.5
113
+ - value: 0.0
114
+ - layer_range: [0, 40]
115
+ model:
116
+ model:
117
+ path: TheBloke/Llama-2-13B-fp16
118
+ ```