kim512 commited on
Commit
52f2f07
1 Parent(s): dddb8d0

Upload README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +109 -0
README.md ADDED
@@ -0,0 +1,109 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ base_model: []
3
+ library_name: transformers
4
+ tags:
5
+ - mergekit
6
+ - merge
7
+ - llama 3
8
+ - 70b
9
+ - arimas
10
+ - story
11
+ - roleplay
12
+ - rp
13
+ ---
14
+
15
+ # EXL2 quants of [ryzen88/Llama-3-70b-Arimas-story-RP-V1.6](https://huggingface.co/ryzen88/Llama-3-70b-Arimas-story-RP-V1.6)
16
+
17
+ [3.00 bits per weight](https://huggingface.co/kim512/Llama-3-70b-Arimas-story-RP-V1.6-3.0bpw-h6-exl2)
18
+ [3.50 bits per weight](https://huggingface.co/kim512/Llama-3-70b-Arimas-story-RP-V1.6-3.5bpw-h6-exl2)
19
+ [4.00 bits per weight](https://huggingface.co/kim512/Llama-3-70b-Arimas-story-RP-V1.6-4.0bpw-h6-exl2)
20
+ [4.50 bits per weight](https://huggingface.co/kim512/Llama-3-70b-Arimas-story-RP-V1.6-4.5bpw-h6-exl2)
21
+ [6.00 bits per weight](https://huggingface.co/kim512/Llama-3-70b-Arimas-story-RP-V1.6-6.0bpw-h6-exl2)
22
+ [8.00 bits per weight](https://huggingface.co/kim512/Llama-3-70b-Arimas-story-RP-V1.6-8.0bpw-h8-exl2)
23
+
24
+ Created using the defaults from exllamav2 1.4.0 convert.py
25
+ 3.0bpw to 6.0bpw head bits = 6
26
+ 8.0bpw head bits = 8
27
+ length = 8192
28
+ dataset rows = 200
29
+ measurement rows = 32
30
+ measurement length = 8192
31
+
32
+
33
+ # model
34
+ Llama-3-70b-Arimas-story-RP-V1.6
35
+
36
+ This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit).
37
+
38
+ ## Merge Details
39
+ I Greatly expanded the amount of models used in this merge, experimented a lot with different idea's.
40
+ This version feels a lot more convincing than V1.5 Hopefully the long context window will also remain strong after Quants.
41
+ Because of the many merges switched back from BFloat to Float.
42
+ Tried breadcrums without the Ties, that went very poorly.
43
+
44
+ ### Merge Method
45
+
46
+ This model was merged using the breadcrumbs_ties merge method using I:\Llama-3-70B-Instruct-Gradient-262k as a base.
47
+
48
+ ### Models Merged
49
+
50
+ The following models were included in the merge:
51
+ * \Smaug-Llama-3-70B-Instruct
52
+ * \Meta-LLama-3-Cat-Smaug-LLama-70b
53
+ * \Meta-LLama-3-Cat-A-LLama-70b
54
+ * \Llama-3-70B-Synthia-v3.5
55
+ * \Llama-3-70B-Instruct-Gradient-524k
56
+ * \Llama-3-70B-Instruct-Gradient-262k
57
+ * \Tess-2.0-Llama-3-70B-v0.2
58
+ * \Llama-3-Lumimaid-70B-v0.1-alt
59
+
60
+ ### Configuration
61
+
62
+ The following YAML configuration was used to produce this model:
63
+
64
+ ```yaml
65
+ models:
66
+ - model: \Llama-3-70B-Instruct-Gradient-262k
67
+ parameters:
68
+ weight: 0.25
69
+ density: 0.90
70
+ gamma: 0.01
71
+ - model: \Meta-LLama-3-Cat-Smaug-LLama-70b
72
+ parameters:
73
+ weight: 0.28
74
+ density: 0.90
75
+ gamma: 0.01
76
+ - model: \Llama-3-Lumimaid-70B-v0.1-alt
77
+ parameters:
78
+ weight: 0.15
79
+ density: 0.90
80
+ gamma: 0.01
81
+ - model: \Tess-2.0-Llama-3-70B-v0.2
82
+ parameters:
83
+ weight: 0.06
84
+ density: 0.90
85
+ gamma: 0.01
86
+ - model: \Smaug-Llama-3-70B-Instruct
87
+ parameters:
88
+ weight: 0.04
89
+ density: 0.90
90
+ gamma: 0.01
91
+ - model: \Llama-3-70B-Synthia-v3.5
92
+ parameters:
93
+ weight: 0.05
94
+ density: 0.90
95
+ gamma: 0.01
96
+ - model: \Llama-3-70B-Instruct-Gradient-524k
97
+ parameters:
98
+ weight: 0.03
99
+ density: 0.90
100
+ gamma: 0.01
101
+ - model: \Meta-LLama-3-Cat-A-LLama-70b
102
+ parameters:
103
+ weight: 0.14
104
+ density: 0.90
105
+ gamma: 0.01
106
+ merge_method: breadcrumbs_ties
107
+ base_model: I:\Llama-3-70B-Instruct-Gradient-262k
108
+ dtype: float16
109
+ ```