TheBloke commited on
Commit
d6d6495
1 Parent(s): a7d4b93

Update for Transformers GPTQ support

Browse files
README.md CHANGED
@@ -4,17 +4,20 @@ license: other
4
  ---
5
 
6
  <!-- header start -->
7
- <div style="width: 100%;">
8
- <img src="https://i.imgur.com/EBdldam.jpg" alt="TheBlokeAI" style="width: 100%; min-width: 400px; display: block; margin: auto;">
 
9
  </div>
10
  <div style="display: flex; justify-content: space-between; width: 100%;">
11
  <div style="display: flex; flex-direction: column; align-items: flex-start;">
12
- <p><a href="https://discord.gg/theblokeai">Chat & support: my new Discord server</a></p>
13
  </div>
14
  <div style="display: flex; flex-direction: column; align-items: flex-end;">
15
- <p><a href="https://www.patreon.com/TheBlokeAI">Want to contribute? TheBloke's Patreon page</a></p>
16
  </div>
17
  </div>
 
 
18
  <!-- header end -->
19
 
20
  # OpenAccess AI Collective's Manticore 13B Chat GPTQ
@@ -146,6 +149,7 @@ It was created with group_size 128 to increase inference accuracy, but without -
146
  * Parameters: Groupsize = 128. Act Order / desc_act = False.
147
 
148
  <!-- footer start -->
 
149
  ## Discord
150
 
151
  For further support, and discussions on these models and AI in general, join us at:
@@ -165,12 +169,15 @@ Donaters will get priority support on any and all AI/LLM/model questions and req
165
  * Patreon: https://patreon.com/TheBlokeAI
166
  * Ko-Fi: https://ko-fi.com/TheBlokeAI
167
 
168
- **Special thanks to**: Luke from CarbonQuill, Aemon Algiz, Dmitriy Samsonov.
 
 
169
 
170
- **Patreon special mentions**: zynix , ya boyyy, Trenton Dambrowitz, Imad Khwaja, Alps Aficionado, chris gileta, John Detwiler, Willem Michiel, RoA, Mano Prime, Rainer Wilmers, Fred von Graf, Matthew Berman, Ghost , Nathan LeClaire, Iucharbius , Ai Maven, Illia Dulskyi, Joseph William Delisle, Space Cruiser, Lone Striker, Karl Bernard, Eugene Pentland, Greatston Gnanesh, Jonathan Leane, Randy H, Pierre Kircher, Willian Hasse, Stephen Murray, Alex , terasurfer , Edmond Seymore, Oscar Rangel, Luke Pendergrass, Asp the Wyvern, Junyu Yang, David Flickinger, Luke, Spiking Neurons AB, subjectnull, Pyrater, Nikolai Manek, senxiiz, Ajan Kanaga, Johann-Peter Hartmann, Artur Olbinski, Kevin Schuppel, Derek Yates, Kalila, K, Talal Aujan, Khalefa Al-Ahmad, Gabriel Puliatti, John Villwock, WelcomeToTheClub, Daniel P. Andersen, Preetika Verma, Deep Realms, Fen Risland, trip7s trip, webtim, Sean Connelly, Michael Levine, Chris McCloskey, biorpg, vamX, Viktor Bowallius, Cory Kujawski.
171
 
172
  Thank you to all my generous patrons and donaters!
173
 
 
 
174
  <!-- footer end -->
175
 
176
  # Original model card: Kaio Ken's SuperHOT 8K
@@ -212,21 +219,21 @@ I trained the LoRA with the following configuration:
212
 
213
  [<img src="https://raw.githubusercontent.com/OpenAccess-AI-Collective/axolotl/main/image/axolotl-badge-web.png" alt="Built with Axolotl" width="200" height="32"/>](https://github.com/OpenAccess-AI-Collective/axolotl)
214
 
215
- Manticore 13B Chat builds on Manticore with new datasets, including a de-duped subset of the Pygmalion dataset. It also removes all Alpaca style prompts using `###` in favor of
216
  chat only style prompts using `USER:`,`ASSISTANT:` as well as [pygmalion/metharme prompting](https://huggingface.co/PygmalionAI/metharme-7b#prompting) using `<|system|>, <|user|> and <|model|>` tokens.
217
 
218
  Questions, comments, feedback, looking to donate, or want to help? Reach out on our [Discord](https://discord.gg/PugNNHAF5r) or email [wing@openaccessaicollective.org](mailto:wing@openaccessaicollective.org)
219
 
220
  # Training Datasets
221
 
222
- Manticore 13B Chat is a Llama 13B model fine-tuned on the following datasets along with the datasets from the original Manticore 13B.
223
 
224
  **Manticore 13B Chat was trained on 25% of the datasets below. The datasets were merged, shuffled, and then sharded into 4 parts.**
225
 
226
  - de-duped pygmalion dataset, filtered down to RP data
227
- - [riddle_sense](https://huggingface.co/datasets/riddle_sense) - instruct augmented
228
  - hellaswag, updated for detailed explanations w 30K+ rows
229
- - [gsm8k](https://huggingface.co/datasets/gsm8k) - instruct augmented
230
  - [ewof/code-alpaca-instruct-unfiltered](https://huggingface.co/datasets/ewof/code-alpaca-instruct-unfiltered)
231
 
232
  Manticore 13B
@@ -258,8 +265,8 @@ Try out the model in HF Spaces. The demo uses a quantized GGML version of the mo
258
 
259
  ## Build
260
 
261
- Manticore was built with [Axolotl](https://github.com/OpenAccess-AI-Collective/axolotl) on 8xA100 80GB
262
- - 3 epochs taking approximately 8 hours. No further epochs will be released.
263
  - The configuration to duplicate this build is provided in this repo's [/config folder](https://huggingface.co/openaccess-ai-collective/manticore-13b/tree/main/configs).
264
 
265
  ## Bias, Risks, and Limitations
 
4
  ---
5
 
6
  <!-- header start -->
7
+ <!-- 200823 -->
8
+ <div style="width: auto; margin-left: auto; margin-right: auto">
9
+ <img src="https://i.imgur.com/EBdldam.jpg" alt="TheBlokeAI" style="width: 100%; min-width: 400px; display: block; margin: auto;">
10
  </div>
11
  <div style="display: flex; justify-content: space-between; width: 100%;">
12
  <div style="display: flex; flex-direction: column; align-items: flex-start;">
13
+ <p style="margin-top: 0.5em; margin-bottom: 0em;"><a href="https://discord.gg/theblokeai">Chat & support: TheBloke's Discord server</a></p>
14
  </div>
15
  <div style="display: flex; flex-direction: column; align-items: flex-end;">
16
+ <p style="margin-top: 0.5em; margin-bottom: 0em;"><a href="https://www.patreon.com/TheBlokeAI">Want to contribute? TheBloke's Patreon page</a></p>
17
  </div>
18
  </div>
19
+ <div style="text-align:center; margin-top: 0em; margin-bottom: 0em"><p style="margin-top: 0.25em; margin-bottom: 0em;">TheBloke's LLM work is generously supported by a grant from <a href="https://a16z.com">andreessen horowitz (a16z)</a></p></div>
20
+ <hr style="margin-top: 1.0em; margin-bottom: 1.0em;">
21
  <!-- header end -->
22
 
23
  # OpenAccess AI Collective's Manticore 13B Chat GPTQ
 
149
  * Parameters: Groupsize = 128. Act Order / desc_act = False.
150
 
151
  <!-- footer start -->
152
+ <!-- 200823 -->
153
  ## Discord
154
 
155
  For further support, and discussions on these models and AI in general, join us at:
 
169
  * Patreon: https://patreon.com/TheBlokeAI
170
  * Ko-Fi: https://ko-fi.com/TheBlokeAI
171
 
172
+ **Special thanks to**: Aemon Algiz.
173
+
174
+ **Patreon special mentions**: Sam, theTransient, Jonathan Leane, Steven Wood, webtim, Johann-Peter Hartmann, Geoffrey Montalvo, Gabriel Tamborski, Willem Michiel, John Villwock, Derek Yates, Mesiah Bishop, Eugene Pentland, Pieter, Chadd, Stephen Murray, Daniel P. Andersen, terasurfer, Brandon Frisco, Thomas Belote, Sid, Nathan LeClaire, Magnesian, Alps Aficionado, Stanislav Ovsiannikov, Alex, Joseph William Delisle, Nikolai Manek, Michael Davis, Junyu Yang, K, J, Spencer Kim, Stefan Sabev, Olusegun Samson, transmissions 11, Michael Levine, Cory Kujawski, Rainer Wilmers, zynix, Kalila, Luke @flexchar, Ajan Kanaga, Mandus, vamX, Ai Maven, Mano Prime, Matthew Berman, subjectnull, Vitor Caleffi, Clay Pascal, biorpg, alfie_i, 阿明, Jeffrey Morgan, ya boyyy, Raymond Fosdick, knownsqashed, Olakabola, Leonard Tan, ReadyPlayerEmma, Enrico Ros, Dave, Talal Aujan, Illia Dulskyi, Sean Connelly, senxiiz, Artur Olbinski, Elle, Raven Klaugh, Fen Risland, Deep Realms, Imad Khwaja, Fred von Graf, Will Dee, usrbinkat, SuperWojo, Alexandros Triantafyllidis, Swaroop Kallakuri, Dan Guido, John Detwiler, Pedro Madruga, Iucharbius, Viktor Bowallius, Asp the Wyvern, Edmond Seymore, Trenton Dambrowitz, Space Cruiser, Spiking Neurons AB, Pyrater, LangChain4j, Tony Hughes, Kacper Wikieł, Rishabh Srivastava, David Ziegler, Luke Pendergrass, Andrey, Gabriel Puliatti, Lone Striker, Sebastain Graf, Pierre Kircher, Randy H, NimbleBox.ai, Vadim, danny, Deo Leter
175
 
 
176
 
177
  Thank you to all my generous patrons and donaters!
178
 
179
+ And thank you again to a16z for their generous grant.
180
+
181
  <!-- footer end -->
182
 
183
  # Original model card: Kaio Ken's SuperHOT 8K
 
219
 
220
  [<img src="https://raw.githubusercontent.com/OpenAccess-AI-Collective/axolotl/main/image/axolotl-badge-web.png" alt="Built with Axolotl" width="200" height="32"/>](https://github.com/OpenAccess-AI-Collective/axolotl)
221
 
222
+ Manticore 13B Chat builds on Manticore with new datasets, including a de-duped subset of the Pygmalion dataset. It also removes all Alpaca style prompts using `###` in favor of
223
  chat only style prompts using `USER:`,`ASSISTANT:` as well as [pygmalion/metharme prompting](https://huggingface.co/PygmalionAI/metharme-7b#prompting) using `<|system|>, <|user|> and <|model|>` tokens.
224
 
225
  Questions, comments, feedback, looking to donate, or want to help? Reach out on our [Discord](https://discord.gg/PugNNHAF5r) or email [wing@openaccessaicollective.org](mailto:wing@openaccessaicollective.org)
226
 
227
  # Training Datasets
228
 
229
+ Manticore 13B Chat is a Llama 13B model fine-tuned on the following datasets along with the datasets from the original Manticore 13B.
230
 
231
  **Manticore 13B Chat was trained on 25% of the datasets below. The datasets were merged, shuffled, and then sharded into 4 parts.**
232
 
233
  - de-duped pygmalion dataset, filtered down to RP data
234
+ - [riddle_sense](https://huggingface.co/datasets/riddle_sense) - instruct augmented
235
  - hellaswag, updated for detailed explanations w 30K+ rows
236
+ - [gsm8k](https://huggingface.co/datasets/gsm8k) - instruct augmented
237
  - [ewof/code-alpaca-instruct-unfiltered](https://huggingface.co/datasets/ewof/code-alpaca-instruct-unfiltered)
238
 
239
  Manticore 13B
 
265
 
266
  ## Build
267
 
268
+ Manticore was built with [Axolotl](https://github.com/OpenAccess-AI-Collective/axolotl) on 8xA100 80GB
269
+ - 3 epochs taking approximately 8 hours. No further epochs will be released.
270
  - The configuration to duplicate this build is provided in this repo's [/config folder](https://huggingface.co/openaccess-ai-collective/manticore-13b/tree/main/configs).
271
 
272
  ## Bias, Risks, and Limitations
config.json CHANGED
@@ -1,29 +1,39 @@
1
  {
2
- "_name_or_path": "/workspace/superhot_process/manticore-13b-chat-pyg/source",
3
- "architectures": [
4
- "LlamaForCausalLM"
5
- ],
6
- "bos_token_id": 1,
7
- "eos_token_id": 2,
8
- "hidden_act": "silu",
9
- "hidden_size": 5120,
10
- "initializer_range": 0.02,
11
- "intermediate_size": 13824,
12
- "max_position_embeddings": 8192,
13
- "max_sequence_length": 2048,
14
- "model_type": "llama",
15
- "num_attention_heads": 40,
16
- "num_hidden_layers": 40,
17
- "pad_token_id": 0,
18
- "rms_norm_eps": 1e-06,
19
- "tie_word_embeddings": false,
20
- "torch_dtype": "float16",
21
- "transformers_version": "4.30.0.dev0",
22
- "use_cache": true,
23
- "vocab_size": 32000,
24
- "auto_map": {
25
- "AutoModel": "modelling_llama.LlamaModel",
26
- "AutoModelForCausalLM": "modelling_llama.LlamaForCausalLM",
27
- "AutoModelForSequenceClassification": "modelling_llama.LlamaForSequenceClassification"
28
- }
 
 
 
 
 
 
 
 
 
 
29
  }
 
1
  {
2
+ "_name_or_path": "/workspace/superhot_process/manticore-13b-chat-pyg/source",
3
+ "architectures": [
4
+ "LlamaForCausalLM"
5
+ ],
6
+ "bos_token_id": 1,
7
+ "eos_token_id": 2,
8
+ "hidden_act": "silu",
9
+ "hidden_size": 5120,
10
+ "initializer_range": 0.02,
11
+ "intermediate_size": 13824,
12
+ "max_position_embeddings": 8192,
13
+ "max_sequence_length": 2048,
14
+ "model_type": "llama",
15
+ "num_attention_heads": 40,
16
+ "num_hidden_layers": 40,
17
+ "pad_token_id": 0,
18
+ "rms_norm_eps": 1e-06,
19
+ "tie_word_embeddings": false,
20
+ "torch_dtype": "float16",
21
+ "transformers_version": "4.30.0.dev0",
22
+ "use_cache": true,
23
+ "vocab_size": 32000,
24
+ "auto_map": {
25
+ "AutoModel": "modelling_llama.LlamaModel",
26
+ "AutoModelForCausalLM": "modelling_llama.LlamaForCausalLM",
27
+ "AutoModelForSequenceClassification": "modelling_llama.LlamaForSequenceClassification"
28
+ },
29
+ "quantization_config": {
30
+ "bits": 4,
31
+ "group_size": 128,
32
+ "damp_percent": 0.01,
33
+ "desc_act": false,
34
+ "sym": true,
35
+ "true_sequential": true,
36
+ "model_file_base_name": "model",
37
+ "quant_method": "gptq"
38
+ }
39
  }
manticore-13b-chat-pyg-superhot-8k-GPTQ-4bit-128g.no-act.order.safetensors → model.safetensors RENAMED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:fa93cb4050e1123760b6e0cd6ceeb1ee00a8a10fdf8bee3de19c061a95860b4e
3
- size 7454797160
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:39a3d45d0c40d4206d8f936608529c72f2c850e2cf5c00de399502c59f2533f7
3
+ size 7454797216
quantize_config.json CHANGED
@@ -1,8 +1,9 @@
1
  {
2
- "bits": 4,
3
- "group_size": 128,
4
- "damp_percent": 0.01,
5
- "desc_act": false,
6
- "sym": true,
7
- "true_sequential": true
 
8
  }
 
1
  {
2
+ "bits": 4,
3
+ "group_size": 128,
4
+ "damp_percent": 0.01,
5
+ "desc_act": false,
6
+ "sym": true,
7
+ "true_sequential": true,
8
+ "model_file_base_name": "model"
9
  }