Hugging Face's logo

QuantFactory
/
Neural-SOVLish-Devil-8B-L3-GGUF

Text Generation
Transformers
GGUF
mergekit
Merge
Eval Results
Inference Endpoints
Model card Files Community
munish0838 commited on
Commit
64f77f7
•
1 Parent(s): 18dca69

Create README.md

Browse files
Files changed (1) hide show
  1. README.md +169 -0
README.md ADDED
@@ -0,0 +1,169 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: cc-by-nc-4.0
3
+ library_name: transformers
4
+ tags:
5
+ - mergekit
6
+ - merge
7
+ base_model: saishf/Neural-SOVLish-Devil-8B-L3
8
+ model-index:
9
+ - name: Neural-SOVLish-Devil-8B-L3
10
+ results:
11
+ - task:
12
+ type: text-generation
13
+ name: Text Generation
14
+ dataset:
15
+ name: AI2 Reasoning Challenge (25-Shot)
16
+ type: ai2_arc
17
+ config: ARC-Challenge
18
+ split: test
19
+ args:
20
+ num_few_shot: 25
21
+ metrics:
22
+ - type: acc_norm
23
+ value: 69.11
24
+ name: normalized accuracy
25
+ source:
26
+ url: >-
27
+ https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=saishf/Neural-SOVLish-Devil-8B-L3
28
+ name: Open LLM Leaderboard
29
+ - task:
30
+ type: text-generation
31
+ name: Text Generation
32
+ dataset:
33
+ name: HellaSwag (10-Shot)
34
+ type: hellaswag
35
+ split: validation
36
+ args:
37
+ num_few_shot: 10
38
+ metrics:
39
+ - type: acc_norm
40
+ value: 84.77
41
+ name: normalized accuracy
42
+ source:
43
+ url: >-
44
+ https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=saishf/Neural-SOVLish-Devil-8B-L3
45
+ name: Open LLM Leaderboard
46
+ - task:
47
+ type: text-generation
48
+ name: Text Generation
49
+ dataset:
50
+ name: MMLU (5-Shot)
51
+ type: cais/mmlu
52
+ config: all
53
+ split: test
54
+ args:
55
+ num_few_shot: 5
56
+ metrics:
57
+ - type: acc
58
+ value: 69.02
59
+ name: accuracy
60
+ source:
61
+ url: >-
62
+ https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=saishf/Neural-SOVLish-Devil-8B-L3
63
+ name: Open LLM Leaderboard
64
+ - task:
65
+ type: text-generation
66
+ name: Text Generation
67
+ dataset:
68
+ name: TruthfulQA (0-shot)
69
+ type: truthful_qa
70
+ config: multiple_choice
71
+ split: validation
72
+ args:
73
+ num_few_shot: 0
74
+ metrics:
75
+ - type: mc2
76
+ value: 59.05
77
+ source:
78
+ url: >-
79
+ https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=saishf/Neural-SOVLish-Devil-8B-L3
80
+ name: Open LLM Leaderboard
81
+ - task:
82
+ type: text-generation
83
+ name: Text Generation
84
+ dataset:
85
+ name: Winogrande (5-shot)
86
+ type: winogrande
87
+ config: winogrande_xl
88
+ split: validation
89
+ args:
90
+ num_few_shot: 5
91
+ metrics:
92
+ - type: acc
93
+ value: 78.3
94
+ name: accuracy
95
+ source:
96
+ url: >-
97
+ https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=saishf/Neural-SOVLish-Devil-8B-L3
98
+ name: Open LLM Leaderboard
99
+ - task:
100
+ type: text-generation
101
+ name: Text Generation
102
+ dataset:
103
+ name: GSM8k (5-shot)
104
+ type: gsm8k
105
+ config: main
106
+ split: test
107
+ args:
108
+ num_few_shot: 5
109
+ metrics:
110
+ - type: acc
111
+ value: 73.09
112
+ name: accuracy
113
+ source:
114
+ url: >-
115
+ https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=saishf/Neural-SOVLish-Devil-8B-L3
116
+ name: Open LLM Leaderboard
117
+ pipeline_tag: text-generation
118
+ ---
119
+ # QuantFactory/Neural-SOVLish-Devil-8B-L3-GGUF
120
+ This is quantized evrsion of [saishf/Neural-SOVLish-Devil-8B-L3](https://huggingface.co/saishf/Neural-SOVLish-Devil-8B-L3) created using llama.cpp
121
+
122
+ ## Model Description
123
+ This is another "SOVL" style merge, this time using [mlabonne/NeuralDaredevil-8B-abliterated](https://huggingface.co/mlabonne/NeuralDaredevil-8B-abliterated).
124
+
125
+ Daredevil is the first abliterated model series i've tried that feels as smart as base llama-3-instruct while also being willing to give instructions to do all kinda of illegal things
126
+
127
+ Neural daredevil is trained further on the original abliterated model, which should result in a better experience in most scenarios. (A bandaid for the damage abliteration causes)
128
+
129
+ This model should do well in rp, I'm yet to test it (waiting for gguf files @_@)
130
+ ### Merge Method
131
+
132
+ This model was merged using the [Model Stock](https://arxiv.org/abs/2403.19522) merge method using [mlabonne/NeuralDaredevil-8B-abliterated](https://huggingface.co/mlabonne/NeuralDaredevil-8B-abliterated) as a base.
133
+
134
+ ### Models Merged
135
+
136
+ The following models were included in the merge:
137
+ * [mlabonne/NeuralDaredevil-8B-abliterated](https://huggingface.co/mlabonne/NeuralDaredevil-8B-abliterated) + [ResplendentAI/BlueMoon_Llama3](https://huggingface.co/ResplendentAI/BlueMoon_Llama3)
138
+ * [mlabonne/NeuralDaredevil-8B-abliterated](https://huggingface.co/mlabonne/NeuralDaredevil-8B-abliterated) + [ResplendentAI/Smarts_Llama3](https://huggingface.co/ResplendentAI/Smarts_Llama3)
139
+ * [mlabonne/NeuralDaredevil-8B-abliterated](https://huggingface.co/mlabonne/NeuralDaredevil-8B-abliterated) + [ResplendentAI/Luna_Llama3](https://huggingface.co/ResplendentAI/Luna_Llama3)
140
+ * [mlabonne/NeuralDaredevil-8B-abliterated](https://huggingface.co/mlabonne/NeuralDaredevil-8B-abliterated) + [ResplendentAI/Aura_Llama3](https://huggingface.co/ResplendentAI/Aura_Llama3)
141
+ * [mlabonne/NeuralDaredevil-8B-abliterated](https://huggingface.co/mlabonne/NeuralDaredevil-8B-abliterated) + [ResplendentAI/RP_Format_QuoteAsterisk_Llama3](https://huggingface.co/ResplendentAI/RP_Format_QuoteAsterisk_Llama3)
142
+
143
+ ### Configuration
144
+
145
+ The following YAML configuration was used to produce this model:
146
+
147
+ ```yaml
148
+ models:
149
+ - model: mlabonne/NeuralDaredevil-8B-abliterated+ResplendentAI/Aura_Llama3
150
+ - model: mlabonne/NeuralDaredevil-8B-abliterated+ResplendentAI/Smarts_Llama3
151
+ - model: mlabonne/NeuralDaredevil-8B-abliterated+ResplendentAI/Luna_Llama3
152
+ - model: mlabonne/NeuralDaredevil-8B-abliterated+ResplendentAI/BlueMoon_Llama3
153
+ - model: mlabonne/NeuralDaredevil-8B-abliterated+ResplendentAI/RP_Format_QuoteAsterisk_Llama3
154
+ merge_method: model_stock
155
+ base_model: mlabonne/NeuralDaredevil-8B-abliterated
156
+ dtype: bfloat16
157
+ ```
158
+ # [Open LLM Leaderboard Evaluation Results](https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard)
159
+ Detailed results can be found [here](https://huggingface.co/datasets/open-llm-leaderboard/details_saishf__Neural-SOVLish-Devil-8B-L3)
160
+
161
+ | Metric |Value|
162
+ |---------------------------------|----:|
163
+ |Avg. |72.22|
164
+ |AI2 Reasoning Challenge (25-Shot)|69.11|
165
+ |HellaSwag (10-Shot) |84.77|
166
+ |MMLU (5-Shot) |69.02|
167
+ |TruthfulQA (0-shot) |59.05|
168
+ |Winogrande (5-shot) |78.30|
169
+ |GSM8k (5-shot) |73.09|