SyedAbdul commited on
Commit
8c0acfa
1 Parent(s): 171f2ed

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +25 -2
README.md CHANGED
@@ -5,6 +5,29 @@ tags:
5
  - mergekit
6
  ---
7
 
8
- # test-7B-slerp
9
 
10
- This model is a merge of the following models made with [mergekit](https://github.com/cg123/mergekit):
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
5
  - mergekit
6
  ---
7
 
8
+ # NeuralPipe-9B-merged
9
 
10
+ This model is a merge of the following models made with [mergekit](https://github.com/cg123/mergekit):
11
+ * [OpenPipe/mistral-ft-optimized-1218](https://huggingface.co/OpenPipe/mistral-ft-optimized-1218)
12
+ * [cognitivecomputations/dolphin-2.6-mistral-7b-dpo](https://huggingface.co/cognitivecomputations/dolphin-2.6-mistral-7b-dpo)
13
+
14
+ ## 🧩 Configuration
15
+
16
+ ```yaml
17
+ slices:
18
+ - sources:
19
+ - model: OpenPipe/mistral-ft-optimized-1218
20
+ layer_range: [0, 32]
21
+ - model: cognitivecomputations/dolphin-2.6-mistral-7b-dpo
22
+ layer_range: [0, 32]
23
+ merge_method: slerp
24
+ base_model: OpenPipe/mistral-ft-optimized-1218
25
+ parameters:
26
+ t:
27
+ - filter: self_attn
28
+ value: [0, 0.5, 0.3, 0.7, 1]
29
+ - filter: mlp
30
+ value: [1, 0.5, 0.7, 0.3, 0]
31
+ - value: 0.5
32
+ dtype: bfloat16
33
+ ```