gmongaras commited on
Commit
4fa6c21
1 Parent(s): 8ebcccf

Upload 6 files

Browse files
README.md CHANGED
@@ -1,3 +1,76 @@
1
  ---
2
- license: openrail
3
  ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
+ license: other
3
  ---
4
+ <!-- header start -->
5
+ <div style="width: 100%;">
6
+ <img src="https://i.imgur.com/EBdldam.jpg" alt="TheBlokeAI" style="width: 100%; min-width: 400px; display: block; margin: auto;">
7
+ </div>
8
+ <div style="display: flex; justify-content: space-between; width: 100%;">
9
+ <div style="display: flex; flex-direction: column; align-items: flex-start;">
10
+ <p><a href="https://discord.gg/Jq4vkcDakD">Chat & support: my new Discord server</a></p>
11
+ </div>
12
+ <div style="display: flex; flex-direction: column; align-items: flex-end;">
13
+ <p><a href="https://www.patreon.com/TheBlokeAI">Want to contribute? TheBloke's Patreon page</a></p>
14
+ </div>
15
+ </div>
16
+ <!-- header end -->
17
+
18
+ # WizardLM: An Instruction-following LLM Using Evol-Instruct
19
+
20
+ These files are the result of merging the [delta weights](https://huggingface.co/victor123/WizardLM) with the original Llama7B model.
21
+
22
+ The code for merging is provided in the [WizardLM official Github repo](https://github.com/nlpxucan/WizardLM).
23
+
24
+ The original WizardLM deltas are in float32, and this results in producing an HF repo that is also float32, and is much larger than a normal 7B Llama model.
25
+
26
+ Therefore for this repo I converted the merged model to float16, to produce a standard size 7B model.
27
+
28
+ This was achieved by running **`model = model.half()`** prior to saving.
29
+
30
+ ## WizardLM-7B HF
31
+
32
+ This repo contains the full unquantised model files in HF format for GPU inference and as a base for quantisation/conversion.
33
+
34
+ ## Other repositories available
35
+
36
+ * [4bit GGML models for CPU inference](https://huggingface.co/TheBloke/wizardLM-7B-GGML)
37
+ * [4bit GPTQ models for GPU inference](https://huggingface.co/TheBloke/wizardLM-7B-GPTQ)
38
+
39
+ <!-- footer start -->
40
+ ## Discord
41
+
42
+ For further support, and discussions on these models and AI in general, join us at:
43
+
44
+ [TheBloke AI's Discord server](https://discord.gg/Jq4vkcDakD)
45
+
46
+ ## Thanks, and how to contribute.
47
+
48
+ Thanks to the [chirper.ai](https://chirper.ai) team!
49
+
50
+ I've had a lot of people ask if they can contribute. I enjoy providing models and helping people, and would love to be able to spend even more time doing it, as well as expanding into new projects like fine tuning/training.
51
+
52
+ If you're able and willing to contribute it will be most gratefully received and will help me to keep providing more models, and to start work on new AI projects.
53
+
54
+ Donaters will get priority support on any and all AI/LLM/model questions and requests, access to a private Discord room, plus other benefits.
55
+
56
+ * Patreon: https://patreon.com/TheBlokeAI
57
+ * Ko-Fi: https://ko-fi.com/TheBlokeAI
58
+
59
+ **Patreon special mentions**: Aemon Algiz, Dmitriy Samsonov, Nathan LeClaire, Trenton Dambrowitz, Mano Prime, David Flickinger, vamX, Nikolai Manek, senxiiz, Khalefa Al-Ahmad, Illia Dulskyi, Jonathan Leane, Talal Aujan, V. Lukas, Joseph William Delisle, Pyrater, Oscar Rangel, Lone Striker, Luke Pendergrass, Eugene Pentland, Sebastain Graf, Johann-Peter Hartman.
60
+
61
+ Thank you to all my generous patrons and donaters!
62
+ <!-- footer end -->
63
+ # Original model info
64
+
65
+ ## Full details in the model's Github page
66
+
67
+ [WizardLM official Github repo](https://github.com/nlpxucan/WizardLM).
68
+
69
+ ## Overview of Evol-Instruct
70
+
71
+ Evol-Instruct is a novel method using LLMs instead of humans to automatically mass-produce open-domain instructions of various difficulty levels and skills range, to improve the performance of LLMs.
72
+
73
+ Although on our complexity-balanced test set, WizardLM-7B outperforms ChatGPT in the high-complexity instructions, it still lag behind ChatGPT on the entire test set, and we also consider WizardLM to still be in a baby state. This repository will continue to improve WizardLM, train on larger scales, add more training data, and innovate more advanced large-model training methods.
74
+
75
+ ![info](https://github.com/nlpxucan/WizardLM/raw/main/imgs/git_overall.png)
76
+ ![info](https://github.com/nlpxucan/WizardLM/raw/main/imgs/git_running.png)
added_tokens.json ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ {
2
+ "[PAD]": 32000
3
+ }
special_tokens_map.json ADDED
@@ -0,0 +1,24 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "bos_token": {
3
+ "content": "<s>",
4
+ "lstrip": false,
5
+ "normalized": true,
6
+ "rstrip": false,
7
+ "single_word": false
8
+ },
9
+ "eos_token": {
10
+ "content": "</s>",
11
+ "lstrip": false,
12
+ "normalized": true,
13
+ "rstrip": false,
14
+ "single_word": false
15
+ },
16
+ "pad_token": "[PAD]",
17
+ "unk_token": {
18
+ "content": "<unk>",
19
+ "lstrip": false,
20
+ "normalized": true,
21
+ "rstrip": false,
22
+ "single_word": false
23
+ }
24
+ }
tokenizer.json ADDED
The diff for this file is too large to render. See raw diff
 
tokenizer.model ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:9e556afd44213b6bd1be2b850ebbbd98f5481437a8021afaf58ee7fb1818d347
3
+ size 499723
tokenizer_config.json ADDED
@@ -0,0 +1,34 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "add_bos_token": true,
3
+ "add_eos_token": false,
4
+ "bos_token": {
5
+ "__type": "AddedToken",
6
+ "content": "<s>",
7
+ "lstrip": false,
8
+ "normalized": true,
9
+ "rstrip": false,
10
+ "single_word": false
11
+ },
12
+ "clean_up_tokenization_spaces": false,
13
+ "eos_token": {
14
+ "__type": "AddedToken",
15
+ "content": "</s>",
16
+ "lstrip": false,
17
+ "normalized": true,
18
+ "rstrip": false,
19
+ "single_word": false
20
+ },
21
+ "model_max_length": 2048,
22
+ "pad_token": null,
23
+ "padding_side": "right",
24
+ "sp_model_kwargs": {},
25
+ "tokenizer_class": "LlamaTokenizer",
26
+ "unk_token": {
27
+ "__type": "AddedToken",
28
+ "content": "<unk>",
29
+ "lstrip": false,
30
+ "normalized": true,
31
+ "rstrip": false,
32
+ "single_word": false
33
+ }
34
+ }