Commit
•
f17a801
1
Parent(s):
54869dc
Update README.md
Browse files
README.md
CHANGED
@@ -15,6 +15,9 @@ tags:
|
|
15 |
This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit).
|
16 |
|
17 |
## Merge Details
|
|
|
|
|
|
|
18 |
### Merge Method
|
19 |
|
20 |
This model was merged using the [TIES](https://arxiv.org/abs/2306.01708) merge method using [TheBloke/Llama-2-13B-fp16](https://huggingface.co/TheBloke/Llama-2-13B-fp16) as a base.
|
@@ -23,7 +26,9 @@ This model was merged using the [TIES](https://arxiv.org/abs/2306.01708) merge m
|
|
23 |
|
24 |
The following models were included in the merge:
|
25 |
* [Masterjp123/SnowyRP-FinalV1-L2-13B](https://huggingface.co/Masterjp123/SnowyRP-FinalV1-L2-13B)
|
26 |
-
* [
|
|
|
|
|
27 |
* [sauce1337/BerrySauce-L2-13b](https://huggingface.co/sauce1337/BerrySauce-L2-13b)
|
28 |
|
29 |
### Configuration
|
@@ -70,3 +75,44 @@ slices:
|
|
70 |
model:
|
71 |
path: TheBloke/Llama-2-13B-fp16
|
72 |
```
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
15 |
This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit).
|
16 |
|
17 |
## Merge Details
|
18 |
+
|
19 |
+
Made as a test model, not sure about quality, probably will not make any quants unless someone finds out it's good and asks.
|
20 |
+
|
21 |
### Merge Method
|
22 |
|
23 |
This model was merged using the [TIES](https://arxiv.org/abs/2306.01708) merge method using [TheBloke/Llama-2-13B-fp16](https://huggingface.co/TheBloke/Llama-2-13B-fp16) as a base.
|
|
|
26 |
|
27 |
The following models were included in the merge:
|
28 |
* [Masterjp123/SnowyRP-FinalV1-L2-13B](https://huggingface.co/Masterjp123/SnowyRP-FinalV1-L2-13B)
|
29 |
+
* [posicube/Llama2-chat-AYB-13B](https://huggingface.co/posicube/Llama2-chat-AYB-13B)
|
30 |
+
* [Sao10K/Stheno-1.8-L2-13B](https://huggingface.co/Sao10K/Stheno-1.8-L2-13B)
|
31 |
+
* [ValiantLabs/ShiningValiantXS](https://huggingface.co/ValiantLabs/ShiningValiantXS)
|
32 |
* [sauce1337/BerrySauce-L2-13b](https://huggingface.co/sauce1337/BerrySauce-L2-13b)
|
33 |
|
34 |
### Configuration
|
|
|
75 |
model:
|
76 |
path: TheBloke/Llama-2-13B-fp16
|
77 |
```
|
78 |
+
for Masterjp123/Snowyrp-V2B-P1
|
79 |
+
```yaml
|
80 |
+
base_model:
|
81 |
+
model:
|
82 |
+
path: TheBloke/Llama-2-13B-fp16
|
83 |
+
dtype: bfloat16
|
84 |
+
merge_method: ties
|
85 |
+
parameters:
|
86 |
+
int8_mask: 1.0
|
87 |
+
normalize: 1.0
|
88 |
+
slices:
|
89 |
+
- sources:
|
90 |
+
- layer_range: [0, 40]
|
91 |
+
model:
|
92 |
+
model:
|
93 |
+
path: Sao10K/Stheno-1.8-L2-13B
|
94 |
+
parameters:
|
95 |
+
density: [1.0, 0.7, 0.1]
|
96 |
+
weight: 1.0
|
97 |
+
- layer_range: [0, 40]
|
98 |
+
model:
|
99 |
+
model:
|
100 |
+
path: ValiantLabs/ShiningValiantXS
|
101 |
+
parameters:
|
102 |
+
density: 0.5
|
103 |
+
weight: [0.0, 0.3, 0.7, 1.0]
|
104 |
+
- layer_range: [0, 40]
|
105 |
+
model:
|
106 |
+
model:
|
107 |
+
path: posicube/Llama2-chat-AYB-13B
|
108 |
+
parameters:
|
109 |
+
density: 0.33
|
110 |
+
weight:
|
111 |
+
- filter: mlp
|
112 |
+
value: 0.5
|
113 |
+
- value: 0.0
|
114 |
+
- layer_range: [0, 40]
|
115 |
+
model:
|
116 |
+
model:
|
117 |
+
path: TheBloke/Llama-2-13B-fp16
|
118 |
+
```
|