DavidAU commited on
Commit
8dd2b06
·
verified ·
1 Parent(s): c559774

Delete README.md

Browse files
Files changed (1) hide show
  1. README.md +0 -49
README.md DELETED
@@ -1,49 +0,0 @@
1
- ---
2
- base_model: []
3
- library_name: transformers
4
- tags:
5
- - mergekit
6
- - merge
7
-
8
- ---
9
- # Nemo-Grand-Gutenburg-bf16-dr
10
-
11
- This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit).
12
-
13
- ## Merge Details
14
- ### Merge Method
15
-
16
- This model was merged using the [DARE](https://arxiv.org/abs/2311.03099) [TIES](https://arxiv.org/abs/2306.01708) merge method using G:/11B/Mistral-Nemo-Instruct-2407-12B as a base.
17
-
18
- ### Models Merged
19
-
20
- The following models were included in the merge:
21
- * G:/11B/Lyra-Gutenberg-mistral-nemo-12B
22
- * G:/11B/mistral-nemo-gutenberg-12B-v4
23
- * G:/11B/Lyra4-Gutenberg-12B
24
-
25
- ### Configuration
26
-
27
- The following YAML configuration was used to produce this model:
28
-
29
- ```yaml
30
- #
31
- # Grand Gutenburg "one step DARK" weights dense.
32
- #
33
-
34
- models:
35
- - model: G:/11B/mistral-nemo-gutenberg-12B-v4
36
- parameters:
37
- weight: [1,1,1,1,1,1,1,1,1,1,0.7,0.3,0,0,0,0,0,0,0.3,0.7,1,1,1,1,0.7,0.3,0,0,0,0,0,0,0,0,0,0,0,0,0,0]
38
- - model: G:/11B/Lyra4-Gutenberg-12B
39
- parameters:
40
- weight: [0,0,0,0,0,0,0,0,0,0,0.3,0.7,1,1,0.5,0.5,0.5,0.5,0,0,0,0,0,0,0.7,0.3,1,1,1,1,0.7,0.3,0,0,0,0,0,0,0,0]
41
- - model: G:/11B/Lyra-Gutenberg-mistral-nemo-12B
42
- parameters:
43
- weight: [0,0,0,0,0,0,0,0,0,0,0,0,0,0,0.5,0.5,0.5,0.5,0.7,0.3,0,0,0,0,0,0,0,0,0,0,0.3,0.7,1,1,1,1,1,1,1,1]
44
- merge_method: dare_ties
45
- base_model: G:/11B/Mistral-Nemo-Instruct-2407-12B
46
- dtype: bfloat16
47
-
48
-
49
- ```