Hugging Face's logo

AzureBlack
/
Nous-Hermes-Llama2-70b-4.5bpw-6h-exl2

Text Generation
Transformers
PyTorch
Safetensors
English
llama
llama-2
self-instruct
distillation
synthetic instruction
text-generation-inference
Inference Endpoints
Model card Files Community
AzureBlack commited on
Commit
89becaf
•
1 Parent(s): dad8fcc

Create README.md

Browse files
Files changed (1) hide show
  1. README.md +195 -0
README.md ADDED
@@ -0,0 +1,195 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ language:
3
+ - en
4
+ tags:
5
+ - llama-2
6
+ - self-instruct
7
+ - distillation
8
+ - synthetic instruction
9
+ license:
10
+ - mit
11
+ ---
12
+
13
+ Exllama 2 version of model created by the work of NousResearch
14
+
15
+ Original Card https://huggingface.co/NousResearch/Nous-Hermes-Llama2-70b
16
+
17
+ # Model Card: Nous-Hermes-Llama2-70b
18
+
19
+ Compute provided by PygmalionAI, thank you! Follow PygmalionAI on Twitter @pygmalion_ai.
20
+
21
+ ## Model Description
22
+
23
+ Nous-Hermes-Llama2-70b is a state-of-the-art language model fine-tuned on over 300,000 instructions. This model was fine-tuned by Nous Research, with Teknium and Emozilla leading the fine tuning process and dataset curation, Pygmalion sponsoring the compute, and several other contributors.
24
+
25
+ This Hermes model uses the exact same dataset as Hermes on Llama-1. This is to ensure consistency between the old Hermes and new, for anyone who wanted to keep Hermes as similar to the old one, just more capable.
26
+
27
+ This model stands out for its long responses, lower hallucination rate, and absence of OpenAI censorship mechanisms in the synthetic training data. The fine-tuning process was performed with a 4096 sequence length on an 8x H100 80GB machine.
28
+
29
+ ## Model Training
30
+
31
+ The model was trained almost entirely on synthetic GPT-4 outputs. Curating high quality GPT-4 datasets enables incredibly high quality in knowledge, task completion, and style.
32
+
33
+ This includes data from diverse sources such as GPTeacher, the general, roleplay v1&2, code instruct datasets, Nous Instruct & PDACTL (unpublished), and several others, detailed further below
34
+
35
+ ## Collaborators
36
+ The model fine-tuning and the datasets were a collaboration of efforts and resources between Teknium, Karan4D, Emozilla, Huemin Art, and Pygmalion AI.
37
+
38
+ Special mention goes to @winglian for assisting in some of the training issues.
39
+
40
+ Huge shoutout and acknowledgement is deserved for all the dataset creators who generously share their datasets openly.
41
+
42
+ Among the contributors of datasets:
43
+ - GPTeacher was made available by Teknium
44
+ - Wizard LM by nlpxucan
45
+ - Nous Research Instruct Dataset was provided by Karan4D and HueminArt.
46
+ - GPT4-LLM and Unnatural Instructions were provided by Microsoft
47
+ - Airoboros dataset by jondurbin
48
+ - Camel-AI's domain expert datasets are from Camel-AI
49
+ - CodeAlpaca dataset by Sahil 2801.
50
+
51
+ If anyone was left out, please open a thread in the community tab.
52
+
53
+ ## Prompt Format
54
+
55
+ The model follows the Alpaca prompt format:
56
+ ```
57
+ ### Instruction:
58
+ <prompt>
59
+
60
+ ### Response:
61
+ <leave a newline blank for model to respond>
62
+
63
+ ```
64
+
65
+ or
66
+
67
+ ```
68
+ ### Instruction:
69
+ <prompt>
70
+
71
+ ### Input:
72
+ <additional context>
73
+
74
+ ### Response:
75
+ <leave a newline blank for model to respond>
76
+
77
+ ```
78
+
79
+ ## Benchmarks:
80
+
81
+ GPT4All Suite:
82
+
83
+ ```
84
+ hf-causal-experimental (pretrained=/home/data/axolotl/Nous-Hermes-Llama2-70b,dtype=float16,use_accelerate=True), limit: None, provide_description: False, num_fewshot: 0, batch_size: None
85
+ | Task |Version| Metric |Value | |Stderr|
86
+ |-------------|------:|--------|-----:|---|-----:|
87
+ |arc_challenge| 0|acc |0.5734|± |0.0145|
88
+ | | |acc_norm|0.6015|± |0.0143|
89
+ |arc_easy | 0|acc |0.8422|± |0.0075|
90
+ | | |acc_norm|0.8253|± |0.0078|
91
+ |boolq | 1|acc |0.8422|± |0.0064|
92
+ |hellaswag | 0|acc |0.6519|± |0.0048|
93
+ | | |acc_norm|0.8363|± |0.0037|
94
+ |openbookqa | 0|acc |0.3880|± |0.0218|
95
+ | | |acc_norm|0.5000|± |0.0224|
96
+ |piqa | 0|acc |0.8313|± |0.0087|
97
+ | | |acc_norm|0.8351|± |0.0087|
98
+ |winogrande | 0|acc |0.7751|± |0.0117|
99
+ ```
100
+
101
+
102
+ BigBench Suite:
103
+ ```
104
+ hf-causal-experimental (pretrained=/home/data/axolotl/Nous-Hermes-Llama2-70b,dtype=float16,use_accelerate=True), limit: None, provide_description: False, num_fewshot: 0, batch_size: None
105
+ | Task |Version| Metric |Value | |Stderr|
106
+ |------------------------------------------------|------:|---------------------|-----:|---|-----:|
107
+ |bigbench_causal_judgement | 0|multiple_choice_grade|0.6579|± |0.0345|
108
+ |bigbench_date_understanding | 0|multiple_choice_grade|0.7344|± |0.0230|
109
+ |bigbench_disambiguation_qa | 0|multiple_choice_grade|0.3023|± |0.0286|
110
+ |bigbench_geometric_shapes | 0|multiple_choice_grade|0.2340|± |0.0224|
111
+ | | |exact_str_match |0.0000|± |0.0000|
112
+ |bigbench_logical_deduction_five_objects | 0|multiple_choice_grade|0.2760|± |0.0200|
113
+ |bigbench_logical_deduction_seven_objects | 0|multiple_choice_grade|0.1871|± |0.0148|
114
+ |bigbench_logical_deduction_three_objects | 0|multiple_choice_grade|0.4467|± |0.0288|
115
+ |bigbench_movie_recommendation | 0|multiple_choice_grade|0.3240|± |0.0210|
116
+ |bigbench_navigate | 0|multiple_choice_grade|0.5000|�� |0.0158|
117
+ |bigbench_reasoning_about_colored_objects | 0|multiple_choice_grade|0.6605|± |0.0106|
118
+ |bigbench_ruin_names | 0|multiple_choice_grade|0.4598|± |0.0236|
119
+ |bigbench_salient_translation_error_detection | 0|multiple_choice_grade|0.2585|± |0.0139|
120
+ |bigbench_snarks | 0|multiple_choice_grade|0.6630|± |0.0352|
121
+ |bigbench_sports_understanding | 0|multiple_choice_grade|0.7394|± |0.0140|
122
+ |bigbench_temporal_sequences | 0|multiple_choice_grade|0.4440|± |0.0157|
123
+ |bigbench_tracking_shuffled_objects_five_objects | 0|multiple_choice_grade|0.2168|± |0.0117|
124
+ |bigbench_tracking_shuffled_objects_seven_objects| 0|multiple_choice_grade|0.1531|± |0.0086|
125
+ |bigbench_tracking_shuffled_objects_three_objects| 0|multiple_choice_grade|0.4467|± |0.0288|
126
+ ```
127
+
128
+ AGIEval:
129
+ ```
130
+ hf-causal-experimental (pretrained=/home/data/axolotl/Nous-Hermes-Llama2-70b,dtype=float16,use_accelerate=True), limit: None, provide_description: False, num_fewshot: 0, batch_size: None
131
+ | Task |Version| Metric |Value | |Stderr|
132
+ |------------------------------|------:|--------|-----:|---|-----:|
133
+ |agieval_aqua_rat | 0|acc |0.2480|± |0.0272|
134
+ | | |acc_norm|0.2362|± |0.0267|
135
+ |agieval_logiqa_en | 0|acc |0.3917|± |0.0191|
136
+ | | |acc_norm|0.3932|± |0.0192|
137
+ |agieval_lsat_ar | 0|acc |0.2217|± |0.0275|
138
+ | | |acc_norm|0.2000|± |0.0264|
139
+ |agieval_lsat_lr | 0|acc |0.5765|± |0.0219|
140
+ | | |acc_norm|0.4922|± |0.0222|
141
+ |agieval_lsat_rc | 0|acc |0.6914|± |0.0282|
142
+ | | |acc_norm|0.6022|± |0.0299|
143
+ |agieval_sat_en | 0|acc |0.8641|± |0.0239|
144
+ | | |acc_norm|0.8204|± |0.0268|
145
+ |agieval_sat_en_without_passage| 0|acc |0.5291|± |0.0349|
146
+ | | |acc_norm|0.4709|± |0.0349|
147
+ |agieval_sat_math | 0|acc |0.4136|± |0.0333|
148
+ | | |acc_norm|0.3455|± |0.0321|
149
+ ```
150
+
151
+ ## Resources for Applied Use Cases:
152
+ Check out LM Studio for a nice chatgpt style interface here: https://lmstudio.ai/
153
+ For an example of a back and forth chatbot using huggingface transformers and discord, check out: https://github.com/teknium1/alpaca-discord
154
+ For an example of a roleplaying discord chatbot, check out this: https://github.com/teknium1/alpaca-roleplay-discordbot
155
+
156
+ ## Future Plans
157
+ We plan to continue to iterate on both more high quality data, and new data filtering techniques to eliminate lower quality data going forward.
158
+
159
+ ## Model Usage
160
+ The model is available for download on Hugging Face. It is suitable for a wide range of language tasks, from generating creative text to understanding and following complex instructions.
161
+
162
+ [<img src="https://raw.githubusercontent.com/OpenAccess-AI-Collective/axolotl/main/image/axolotl-badge-web.png" alt="Built with Axolotl" width="200" height="32"/>](https://github.com/OpenAccess-AI-Collective/axolotl)
163
+
164
+
165
+ ## Training procedure
166
+
167
+
168
+ The following `bitsandbytes` quantization config was used during training:
169
+ - quant_method: bitsandbytes
170
+ - load_in_8bit: False
171
+ - load_in_4bit: True
172
+ - llm_int8_threshold: 6.0
173
+ - llm_int8_skip_modules: None
174
+ - llm_int8_enable_fp32_cpu_offload: False
175
+ - llm_int8_has_fp16_weight: False
176
+ - bnb_4bit_quant_type: nf4
177
+ - bnb_4bit_use_double_quant: True
178
+ - bnb_4bit_compute_dtype: bfloat16
179
+
180
+ The following `bitsandbytes` quantization config was used during training:
181
+ - quant_method: bitsandbytes
182
+ - load_in_8bit: False
183
+ - load_in_4bit: True
184
+ - llm_int8_threshold: 6.0
185
+ - llm_int8_skip_modules: None
186
+ - llm_int8_enable_fp32_cpu_offload: False
187
+ - llm_int8_has_fp16_weight: False
188
+ - bnb_4bit_quant_type: nf4
189
+ - bnb_4bit_use_double_quant: True
190
+ - bnb_4bit_compute_dtype: bfloat16
191
+ ### Framework versions
192
+
193
+ - PEFT 0.5.0.dev0
194
+
195
+ - PEFT 0.5.0.dev0