rombodawg commited on
Commit
54eea28
1 Parent(s): b04ab69

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +25 -0
README.md CHANGED
@@ -1,3 +1,28 @@
1
  ---
2
  license: apache-2.0
3
  ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
  license: apache-2.0
3
  ---
4
+ This is a test merge of some Gemma-7b finetunes using task_arithmatic. So far its not working, but this is uploaded for testing and diagnosing purposes.
5
+
6
+ Merge config:
7
+ ```yaml
8
+ models:
9
+ - model: gemma-7b-it-fp16
10
+ parameters:
11
+ weight: 1
12
+ - model: CorticalStack_gemma-7b-ultrachat-sft
13
+ parameters:
14
+ weight: 1
15
+ - model: cloudyu_google-gemma-7b-it-dpo-v1
16
+ parameters:
17
+ weight: 1
18
+ - model: abideen_gemma-7b-openhermes
19
+ parameters:
20
+ weight: 1
21
+ merge_method: task_arithmetic
22
+ base_model: gemma-7b-base
23
+ parameters:
24
+ normalize: true
25
+ int8_mask: true
26
+ dtype: float16
27
+
28
+ ```