FPHam commited on
Commit
be23ac4
1 Parent(s): f035c56

Upload 10 files

Browse files
Karen-The-Editor-GPTQ-4bit-128g.no-act.order.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:4268c96414b21c159a97bce0759e3870a5fd128a5069ac539a04b163a0579b1a
3
+ size 7454797160
README.md ADDED
@@ -0,0 +1,134 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ inference: false
3
+ license: other
4
+ ---
5
+
6
+ <!-- header start -->
7
+ <div style="width: 100%;">
8
+ <img src="https://i.imgur.com/EBdldam.jpg" alt="TheBlokeAI" style="width: 100%; min-width: 400px; display: block; margin: auto;">
9
+ </div>
10
+ <div style="display: flex; justify-content: space-between; width: 100%;">
11
+ <div style="display: flex; flex-direction: column; align-items: flex-start;">
12
+ <p><a href="https://discord.gg/UBgz4VXf">Chat & support: my new Discord server</a></p>
13
+ </div>
14
+ <div style="display: flex; flex-direction: column; align-items: flex-end;">
15
+ <p><a href="https://www.patreon.com/TheBlokeAI">Want to contribute? TheBloke's Patreon page</a></p>
16
+ </div>
17
+ </div>
18
+ <!-- header end -->
19
+
20
+ # FPHam's Karen The Editor 13B GPTQ
21
+
22
+ These files are GPTQ 4bit model files for [FPHam's Karen The Editor 13B](https://huggingface.co/FPHam/Karen_theEditor_13b_HF).
23
+
24
+ It is the result of quantising to 4bit using [GPTQ-for-LLaMa](https://github.com/qwopqwop200/GPTQ-for-LLaMa).
25
+
26
+ ## Other repositories available
27
+
28
+ * [4-bit GPTQ models for GPU inference](https://huggingface.co/TheBloke/Karen_theEditor_13B-GPTQ)
29
+ * [4-bit, 5-bit and 8-bit GGML models for CPU(+GPU) inference](https://huggingface.co/TheBloke/Karen_theEditor_13B-GGML)
30
+ * [Unquantised fp16 model in pytorch format, for GPU inference and for further conversions](https://huggingface.co/FPHam/Karen_theEditor_13b_HF
31
+
32
+ ## Prompt template
33
+
34
+ ```
35
+ USER: Edit the following for spelling and grammar mistakes:
36
+ ASSISTANT:
37
+ ```
38
+
39
+ ## How to easily download and use this model in text-generation-webui
40
+
41
+ ### Downloading the model
42
+
43
+ 1. Click the **Model tab**.
44
+ 2. Under **Download custom model or LoRA**, enter `TheBloke/Karen_theEditor_13B-GPTQ`.
45
+ 3. Click **Download**.
46
+ 4. Wait until it says it's finished downloading.
47
+ 5. Untick "Autoload model"
48
+ 6. Click the **Refresh** icon next to **Model** in the top left.
49
+
50
+ ### To use with AutoGPTQ (if installed)
51
+
52
+ 1. In the **Model drop-down**: choose the model you just downloaded, `Karen_theEditor_13B-GPTQ`.
53
+ 2. Under **GPTQ**, tick **AutoGPTQ**.
54
+ 3. Click **Save settings for this model** in the top right.
55
+ 4. Click **Reload the Model** in the top right.
56
+ 5. Once it says it's loaded, click the **Text Generation tab** and enter a prompt!
57
+
58
+ ### To use with GPTQ-for-LLaMa
59
+
60
+ 1. In the **Model drop-down**: choose the model you just downloaded, `Karen_theEditor_13B-GPTQ`.
61
+ 2. If you see an error in the bottom right, ignore it - it's temporary.
62
+ 3. Fill out the `GPTQ parameters` on the right: `Bits = 4`, `Groupsize = 128`, `model_type = Llama`
63
+ 4. Click **Save settings for this model** in the top right.
64
+ 5. Click **Reload the Model** in the top right.
65
+ 6. Once it says it's loaded, click the **Text Generation tab** and enter a prompt!
66
+
67
+ ## Provided files
68
+
69
+ **Karen-The-Editor-GPTQ-4bit-128g.no-act.order.safetensors**
70
+
71
+ This will work with all versions of GPTQ-for-LLaMa, and with AutoGPTQ.
72
+
73
+ It was created with
74
+
75
+ * `Karen-The-Editor-GPTQ-4bit-128g.no-act.order.safetensors`
76
+ * Works with all versions of GPTQ-for-LLaMa code, both Triton and CUDA branches
77
+ * Works with AutoGPTQ
78
+ * Works with text-generation-webui one-click-installers
79
+ * Parameters: Groupsize = 128. Act Order / desc_act = False.
80
+
81
+ <!-- footer start -->
82
+ ## Discord
83
+
84
+ For further support, and discussions on these models and AI in general, join us at:
85
+
86
+ [TheBloke AI's Discord server](https://discord.gg/UBgz4VXf)
87
+
88
+ ## Thanks, and how to contribute.
89
+
90
+ Thanks to the [chirper.ai](https://chirper.ai) team!
91
+
92
+ I've had a lot of people ask if they can contribute. I enjoy providing models and helping people, and would love to be able to spend even more time doing it, as well as expanding into new projects like fine tuning/training.
93
+
94
+ If you're able and willing to contribute it will be most gratefully received and will help me to keep providing more models, and to start work on new AI projects.
95
+
96
+ Donaters will get priority support on any and all AI/LLM/model questions and requests, access to a private Discord room, plus other benefits.
97
+
98
+ * Patreon: https://patreon.com/TheBlokeAI
99
+ * Ko-Fi: https://ko-fi.com/TheBlokeAI
100
+
101
+ **Patreon special mentions**: Aemon Algiz; Dmitiry Samsonov; Jonathan Leane; Illia Dulskyi; Khalefa Al-Ahmad; Nikolai Manek; senxiiz; Talal Aujan; vamX; Eugene Pentland; Lone Striker; Luke Pendergrass; Johann-Peter Hartmann.
102
+
103
+ Thank you to all my generous patrons and donaters.
104
+ <!-- footer end -->
105
+
106
+ # Original model card: FPHam's Karen The Editor 13B
107
+
108
+ ## Karen is an editor for your fiction.
109
+
110
+ She fixes grammar and wording issues, but doesn't necessary start rewording everything like ChatGPT into a corporate talk. So it should keep the style intact.
111
+
112
+ Based on LLAMA 13b and Wizard-Vucna-uncensored finetune, then finetuned with about 20k grammar examples (bad grammar/good grammar).
113
+
114
+ Karen gets triggered by this prompt (pun intended):
115
+
116
+ ```
117
+ USER: Edit the following for spelling and grammar mistakes:
118
+ ASSISTANT:
119
+ ```
120
+
121
+ Feed it a paragraph at a time - that's where she works best - a few sentences.
122
+
123
+ Here is my instruct yaml for oobabooga webui (in Instruct mode):
124
+
125
+ ```
126
+ user: "USER: Edit the following for spelling and grammar mistakes: "
127
+ bot: "ASSISTANT:"
128
+ turn_template: "<|user|> <|user-message|>\n<|bot|> <|bot-message|></s>\n"
129
+ context: "Assistant is a meticulous senior editor with a specialization in editing fictional stories. When given text, Assistant detects and corrects grammatical errors, including subject-verb agreement, tense consistency, punctuation, capitalization, use of correct articles and correct present perfect and past perfect tense.\n"
130
+ ```
131
+
132
+ I am not sure to what extend the context part has any effect. Doesn't hurt though.
133
+
134
+ This is a version 0.2 and finetuned just so it retains the underlaying Vicuna qualities.
config.json ADDED
@@ -0,0 +1,24 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "_name_or_path": "H:\\oobabooga_windows\\text-generation-webui\\models\\TheBloke_Wizard-Vicuna-13B-Uncensored-HF",
3
+ "architectures": [
4
+ "LlamaForCausalLM"
5
+ ],
6
+ "bos_token_id": 1,
7
+ "eos_token_id": 2,
8
+ "hidden_act": "silu",
9
+ "hidden_size": 5120,
10
+ "initializer_range": 0.02,
11
+ "intermediate_size": 13824,
12
+ "max_position_embeddings": 2048,
13
+ "max_sequence_length": 2048,
14
+ "model_type": "llama",
15
+ "num_attention_heads": 40,
16
+ "num_hidden_layers": 40,
17
+ "pad_token_id": 0,
18
+ "rms_norm_eps": 1e-06,
19
+ "tie_word_embeddings": false,
20
+ "torch_dtype": "float16",
21
+ "transformers_version": "4.30.0.dev0",
22
+ "use_cache": true,
23
+ "vocab_size": 32000
24
+ }
generation_config.json ADDED
@@ -0,0 +1,7 @@
 
 
 
 
 
 
 
 
1
+ {
2
+ "_from_model_config": true,
3
+ "bos_token_id": 1,
4
+ "eos_token_id": 2,
5
+ "pad_token_id": 0,
6
+ "transformers_version": "4.30.0.dev0"
7
+ }
huggingface-metadata.txt ADDED
@@ -0,0 +1,6 @@
 
 
 
 
 
 
 
1
+ url: https://huggingface.co/TheBloke/Karen_theEditor_13B-GPTQ
2
+ branch: main
3
+ download date: 2023-06-03 03:34:33
4
+ sha256sum:
5
+ 4268c96414b21c159a97bce0759e3870a5fd128a5069ac539a04b163a0579b1a Karen-The-Editor-GPTQ-4bit-128g.no-act.order.safetensors
6
+ 9e556afd44213b6bd1be2b850ebbbd98f5481437a8021afaf58ee7fb1818d347 tokenizer.model
quantize_config.json ADDED
@@ -0,0 +1,8 @@
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "bits": 4,
3
+ "group_size": 128,
4
+ "damp_percent": 0.01,
5
+ "desc_act": false,
6
+ "sym": true,
7
+ "true_sequential": true
8
+ }
special_tokens_map.json ADDED
@@ -0,0 +1,24 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "bos_token": {
3
+ "content": "<s>",
4
+ "lstrip": false,
5
+ "normalized": true,
6
+ "rstrip": false,
7
+ "single_word": false
8
+ },
9
+ "eos_token": {
10
+ "content": "</s>",
11
+ "lstrip": false,
12
+ "normalized": true,
13
+ "rstrip": false,
14
+ "single_word": false
15
+ },
16
+ "pad_token": "<unk>",
17
+ "unk_token": {
18
+ "content": "<unk>",
19
+ "lstrip": false,
20
+ "normalized": true,
21
+ "rstrip": false,
22
+ "single_word": false
23
+ }
24
+ }
tokenizer.json ADDED
The diff for this file is too large to render. See raw diff
 
tokenizer.model ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:9e556afd44213b6bd1be2b850ebbbd98f5481437a8021afaf58ee7fb1818d347
3
+ size 499723
tokenizer_config.json ADDED
@@ -0,0 +1,34 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "add_bos_token": true,
3
+ "add_eos_token": false,
4
+ "bos_token": {
5
+ "__type": "AddedToken",
6
+ "content": "<s>",
7
+ "lstrip": false,
8
+ "normalized": true,
9
+ "rstrip": false,
10
+ "single_word": false
11
+ },
12
+ "clean_up_tokenization_spaces": false,
13
+ "eos_token": {
14
+ "__type": "AddedToken",
15
+ "content": "</s>",
16
+ "lstrip": false,
17
+ "normalized": true,
18
+ "rstrip": false,
19
+ "single_word": false
20
+ },
21
+ "model_max_length": 2048,
22
+ "pad_token": null,
23
+ "padding_side": "right",
24
+ "sp_model_kwargs": {},
25
+ "tokenizer_class": "LlamaTokenizer",
26
+ "unk_token": {
27
+ "__type": "AddedToken",
28
+ "content": "<unk>",
29
+ "lstrip": false,
30
+ "normalized": true,
31
+ "rstrip": false,
32
+ "single_word": false
33
+ }
34
+ }