solankibhargav commited on
Commit
747ec4d
1 Parent(s): d9fac60

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +6 -6
README.md CHANGED
@@ -4,33 +4,33 @@ tags:
4
  - merge
5
  - mergekit
6
  - lazymergekit
7
- - openchat/openchat-3.5-0106
8
  - machinists/Mistral-7B-SQL
9
  ---
10
 
11
  # haLLAwa2
12
 
13
  haLLAwa2 is a merge of the following models using [mergekit](https://github.com/cg123/mergekit):
14
- * [openchat/openchat-3.5-0106](https://huggingface.co/openchat/openchat-3.5-0106)
15
- * [machinists/Mistral-7B-SQL](https://huggingface.co/machinists/Mistral-7B-SQL)
16
 
17
  ## 🧩 Configuration
18
 
19
  \```yaml
20
  slices:
21
  - sources:
22
- - model: openchat/openchat-3.5-0106
23
  layer_range: [0, 32]
24
  - model: machinists/Mistral-7B-SQL
25
  layer_range: [0, 32]
 
26
  merge_method: slerp
27
- base_model: openchat/openchat-3.5-0106
28
  parameters:
29
  t:
30
  - filter: self_attn
31
  value: [0, 0.5, 0.3, 0.7, 1]
32
  - filter: mlp
33
  value: [1, 0.5, 0.7, 0.3, 0]
34
- - value: 0.5
35
  dtype: bfloat16
36
  \```
 
4
  - merge
5
  - mergekit
6
  - lazymergekit
7
+ - OpenPipe/mistral-ft-optimized-1227
8
  - machinists/Mistral-7B-SQL
9
  ---
10
 
11
  # haLLAwa2
12
 
13
  haLLAwa2 is a merge of the following models using [mergekit](https://github.com/cg123/mergekit):
14
+
 
15
 
16
  ## 🧩 Configuration
17
 
18
  \```yaml
19
  slices:
20
  - sources:
21
+ - model: OpenPipe/mistral-ft-optimized-1227
22
  layer_range: [0, 32]
23
  - model: machinists/Mistral-7B-SQL
24
  layer_range: [0, 32]
25
+
26
  merge_method: slerp
27
+ base_model: OpenPipe/mistral-ft-optimized-1227
28
  parameters:
29
  t:
30
  - filter: self_attn
31
  value: [0, 0.5, 0.3, 0.7, 1]
32
  - filter: mlp
33
  value: [1, 0.5, 0.7, 0.3, 0]
34
+ - value: 0.5 # fallback for rest of tensors
35
  dtype: bfloat16
36
  \```