asiansoul commited on
Commit
a7972d8
1 Parent(s): 0fafd22

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +103 -5
README.md CHANGED
@@ -1,5 +1,103 @@
1
- ---
2
- license: other
3
- license_name: other
4
- license_link: LICENSE
5
- ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ base_model:
3
+ - NousResearch/Hermes-2-Pro-Llama-3-8B
4
+ - cognitivecomputations/dolphin-2.9-llama3-8b
5
+ - NousResearch/Meta-Llama-3-8B
6
+ - winglian/llama-3-8b-256k-PoSE
7
+ - maum-ai/Llama-3-MAAL-8B-Instruct-v0.1
8
+ - asiansoul/Llama-3-Open-Ko-Linear-8B
9
+ - NousResearch/Meta-Llama-3-8B-Instruct
10
+ - nvidia/Llama3-ChatQA-1.5-8B
11
+ - Danielbrdz/Barcenas-Llama3-8b-ORPO
12
+ - aaditya/Llama3-OpenBioLLM-8B
13
+ library_name: transformers
14
+ tags:
15
+ - mergekit
16
+ - merge
17
+ - llama
18
+
19
+ ---
20
+ # YACHT-Llama-3-Ko-8B
21
+
22
+ [![DALL-E Yacht](https://i.ibb.co/hHr5xnh/DALL-E-2024-05-05-11-57-02-A-futuristic-yacht-boat-on-a-calm-ocean-at-dawn-featuring-sleek-curves-an.png)](https://ibb.co/92BXmfz)
23
+
24
+ ## Merge Details
25
+ ### Merge Method
26
+
27
+ This model was merged using the [DARE](https://arxiv.org/abs/2311.03099) [TIES](https://arxiv.org/abs/2306.01708) merge method using [NousResearch/Meta-Llama-3-8B](https://huggingface.co/NousResearch/Meta-Llama-3-8B) as a base.
28
+
29
+ ### Models Merged
30
+
31
+ The following models were included in the merge:
32
+ * [NousResearch/Hermes-2-Pro-Llama-3-8B](https://huggingface.co/NousResearch/Hermes-2-Pro-Llama-3-8B)
33
+ * [cognitivecomputations/dolphin-2.9-llama3-8b](https://huggingface.co/cognitivecomputations/dolphin-2.9-llama3-8b)
34
+ * [winglian/llama-3-8b-256k-PoSE](https://huggingface.co/winglian/llama-3-8b-256k-PoSE)
35
+ * [maum-ai/Llama-3-MAAL-8B-Instruct-v0.1](https://huggingface.co/maum-ai/Llama-3-MAAL-8B-Instruct-v0.1)
36
+ * [asiansoul/Llama-3-Open-Ko-Linear-8B](https://huggingface.co/asiansoul/Llama-3-Open-Ko-Linear-8B)
37
+ * [NousResearch/Meta-Llama-3-8B-Instruct](https://huggingface.co/NousResearch/Meta-Llama-3-8B-Instruct)
38
+ * [nvidia/Llama3-ChatQA-1.5-8B](https://huggingface.co/nvidia/Llama3-ChatQA-1.5-8B)
39
+ * [Danielbrdz/Barcenas-Llama3-8b-ORPO](https://huggingface.co/Danielbrdz/Barcenas-Llama3-8b-ORPO)
40
+ * [aaditya/Llama3-OpenBioLLM-8B](https://huggingface.co/aaditya/Llama3-OpenBioLLM-8B)
41
+
42
+ ### Configuration
43
+
44
+ The following YAML configuration was used to produce this model:
45
+
46
+ ```yaml
47
+ models:
48
+ - model: NousResearch/Meta-Llama-3-8B
49
+ # Base model providing a general foundation without specific parameters
50
+
51
+ - model: NousResearch/Meta-Llama-3-8B-Instruct
52
+ parameters:
53
+ density: 0.60
54
+ weight: 0.25
55
+
56
+ - model: winglian/llama-3-8b-256k-PoSE
57
+ parameters:
58
+ density: 0.55
59
+ weight: 0.15
60
+
61
+ - model: nvidia/Llama3-ChatQA-1.5-8B
62
+ parameters:
63
+ density: 0.55
64
+ weight: 0.1
65
+
66
+ - model: asiansoul/Llama-3-Open-Ko-Linear-8B
67
+ parameters:
68
+ density: 0.55
69
+ weight: 0.2
70
+
71
+ - model: maum-ai/Llama-3-MAAL-8B-Instruct-v0.1
72
+ parameters:
73
+ density: 0.55
74
+ weight: 0.1
75
+
76
+ - model: NousResearch/Hermes-2-Pro-Llama-3-8B
77
+ parameters:
78
+ density: 0.55
79
+ weight: 0.1
80
+
81
+ - model: cognitivecomputations/dolphin-2.9-llama3-8b
82
+ parameters:
83
+ density: 0.55
84
+ weight: 0.05
85
+
86
+ - model: Danielbrdz/Barcenas-Llama3-8b-ORPO
87
+ parameters:
88
+ density: 0.55
89
+ weight: 0.05
90
+
91
+ - model: aaditya/Llama3-OpenBioLLM-8B
92
+ parameters:
93
+ density: 0.55
94
+ weight: 0.1
95
+
96
+ merge_method: dare_ties
97
+ base_model: NousResearch/Meta-Llama-3-8B
98
+ parameters:
99
+ int8_mask: true
100
+ dtype: bfloat16
101
+
102
+
103
+ ```