Azazelle commited on
Commit
36c2dfb
1 Parent(s): bc7abd1

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +27 -0
README.md CHANGED
@@ -1,3 +1,30 @@
1
  ---
 
 
 
 
2
  license: cc-by-4.0
3
  ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
+ pipeline_tag: text-generation
3
+ tags:
4
+ - mistral
5
+ - merge
6
  license: cc-by-4.0
7
  ---
8
+ # Model Card for xDAN-SlimOrca
9
+ <!-- Provide a quick summary of what the model is/does. -->
10
+ Slerp merge of Silicon-Maid-7B, piano-medley-7b, xDAN-L1-Chat-RL-v1, and mistral-7b-slimorcaboros.
11
+
12
+ .yaml file for mergekit
13
+ ```.yaml:
14
+ slices:
15
+ - sources:
16
+ - model: Azazelle/Silicon-Medley
17
+ layer_range: [0, 32]
18
+ - model: Azazelle/xDAN-SlimOrca
19
+ layer_range: [0, 32]
20
+ merge_method: slerp
21
+ base_model: mistralai/Mistral-7B-v0.1
22
+ parameters:
23
+ t:
24
+ - filter: self_attn
25
+ value: [0.19, 0.59, 0.43, 0.76, 1]
26
+ - filter: mlp
27
+ value: [0.81, 0.41, 0.57, 0.24, 0]
28
+ - value: 0.5 # fallback for rest of tensors
29
+ dtype: float16
30
+ ```