AALF commited on
Commit
6af538f
1 Parent(s): 554815b

Delete README.md

Browse files
Files changed (1) hide show
  1. README.md +0 -61
README.md DELETED
@@ -1,61 +0,0 @@
1
- ---
2
- base_model: []
3
- library_name: transformers
4
- tags:
5
- - mergekit
6
- - merge
7
-
8
- ---
9
- # FuseChat-7B-Ties-VaRM-comb1-top0.1
10
-
11
- This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit).
12
-
13
- ## Merge Details
14
- ### Merge Method
15
-
16
- This model was merged using the ties_varm merge method using /data/wanfq/fusechat/models/openchat_3.5 as a base.
17
-
18
- ### Models Merged
19
-
20
- The following models were included in the merge:
21
- * /data/wanfq/agno_fusion/fusechat_models/OpenChat-3.5-7B-Mixtral
22
- * /data/wanfq/agno_fusion/fusechat_models/OpenChat-3.5-7B-NHSolar
23
- * /data/wanfq/agno_fusion/fusechat_models/OpenChat-3.5-7B-Qwen
24
- * /data/wanfq/agno_fusion/fusechat_models/OpenChat-3.5-7B-internlm
25
- * /data/wanfq/agno_fusion/fusechat_models/OpenChat-3.5-7B-Starling
26
-
27
- ### Configuration
28
-
29
- The following YAML configuration was used to produce this model:
30
-
31
- ```yaml
32
- models:
33
- - model: /data/wanfq/fusechat/models/openchat_3.5
34
- # no parameters necessary for base model
35
- - model: /data/wanfq/agno_fusion/fusechat_models/OpenChat-3.5-7B-internlm
36
- parameters:
37
- density: 0.1 # density gradient
38
- weight: 0.2
39
- - model: /data/wanfq/agno_fusion/fusechat_models/OpenChat-3.5-7B-Qwen
40
- parameters:
41
- density: 0.5
42
- weight: 0.2 # weight gradient
43
- - model: /data/wanfq/agno_fusion/fusechat_models/OpenChat-3.5-7B-Mixtral
44
- parameters:
45
- density: 0.5
46
- weight: 0.2 # weight gradient
47
- - model: /data/wanfq/agno_fusion/fusechat_models/OpenChat-3.5-7B-Starling
48
- parameters:
49
- density: 0.5
50
- weight: 0.2 # weight gradient
51
- - model: /data/wanfq/agno_fusion/fusechat_models/OpenChat-3.5-7B-NHSolar
52
- parameters:
53
- density: 0.5
54
- weight: 0.2 # weight gradient
55
- merge_method: ties_varm
56
- base_model: /data/wanfq/fusechat/models/openchat_3.5
57
- parameters:
58
- normalize: true
59
- dtype: bfloat16
60
-
61
- ```