Hugging Face's logo

KingNish
/
WizardLM2-2bit

Text Generation
Transformers
Safetensors
GGUF
mixtral
quantized
2-bit
GGUF
mistral
arxiv:2304.12244
arxiv:2306.08568
arxiv:2308.09583
Inference Endpoints
text-generation-inference
conversational
Model card Files Community
3
KingNish MaziyarPanahi commited on
Commit
47ef9a4
•
0 Parent(s):

Duplicate from MaziyarPanahi/WizardLM-2-8x22B-GGUF

Browse files

Co-authored-by: Maziyar Panahi <MaziyarPanahi@users.noreply.huggingface.co>

This view is limited to 50 files because it contains too many changes.   See raw diff
Files changed (50) hide show
  1. .gitattributes +97 -0
  2. README.md +74 -0
  3. WizardLM-2-8x22B.IQ1_M.gguf +3 -0
  4. WizardLM-2-8x22B.IQ1_S.gguf +3 -0
  5. WizardLM-2-8x22B.IQ3_XS-00001-of-00005.gguf +3 -0
  6. WizardLM-2-8x22B.IQ3_XS-00002-of-00005.gguf +3 -0
  7. WizardLM-2-8x22B.IQ3_XS-00003-of-00005.gguf +3 -0
  8. WizardLM-2-8x22B.IQ3_XS-00004-of-00005.gguf +3 -0
  9. WizardLM-2-8x22B.IQ3_XS-00005-of-00005.gguf +3 -0
  10. WizardLM-2-8x22B.IQ4_XS-00001-of-00005.gguf +3 -0
  11. WizardLM-2-8x22B.IQ4_XS-00002-of-00005.gguf +3 -0
  12. WizardLM-2-8x22B.IQ4_XS-00003-of-00005.gguf +3 -0
  13. WizardLM-2-8x22B.IQ4_XS-00004-of-00005.gguf +3 -0
  14. WizardLM-2-8x22B.IQ4_XS-00005-of-00005.gguf +3 -0
  15. WizardLM-2-8x22B.Q2_K-00001-of-00005.gguf +3 -0
  16. WizardLM-2-8x22B.Q2_K-00002-of-00005.gguf +3 -0
  17. WizardLM-2-8x22B.Q2_K-00003-of-00005.gguf +3 -0
  18. WizardLM-2-8x22B.Q2_K-00004-of-00005.gguf +3 -0
  19. WizardLM-2-8x22B.Q2_K-00005-of-00005.gguf +3 -0
  20. WizardLM-2-8x22B.Q3_K_L-00001-of-00005.gguf +3 -0
  21. WizardLM-2-8x22B.Q3_K_L-00002-of-00005.gguf +3 -0
  22. WizardLM-2-8x22B.Q3_K_L-00003-of-00005.gguf +3 -0
  23. WizardLM-2-8x22B.Q3_K_L-00004-of-00005.gguf +3 -0
  24. WizardLM-2-8x22B.Q3_K_L-00005-of-00005.gguf +3 -0
  25. WizardLM-2-8x22B.Q3_K_M-00001-of-00005.gguf +3 -0
  26. WizardLM-2-8x22B.Q3_K_M-00002-of-00005.gguf +3 -0
  27. WizardLM-2-8x22B.Q3_K_M-00003-of-00005.gguf +3 -0
  28. WizardLM-2-8x22B.Q3_K_M-00004-of-00005.gguf +3 -0
  29. WizardLM-2-8x22B.Q3_K_M-00005-of-00005.gguf +3 -0
  30. WizardLM-2-8x22B.Q3_K_S-00001-of-00005.gguf +3 -0
  31. WizardLM-2-8x22B.Q3_K_S-00002-of-00005.gguf +3 -0
  32. WizardLM-2-8x22B.Q3_K_S-00003-of-00005.gguf +3 -0
  33. WizardLM-2-8x22B.Q3_K_S-00004-of-00005.gguf +3 -0
  34. WizardLM-2-8x22B.Q3_K_S-00005-of-00005.gguf +3 -0
  35. WizardLM-2-8x22B.Q4_K_M-00001-of-00005.gguf +3 -0
  36. WizardLM-2-8x22B.Q4_K_M-00002-of-00005.gguf +3 -0
  37. WizardLM-2-8x22B.Q4_K_M-00003-of-00005.gguf +3 -0
  38. WizardLM-2-8x22B.Q4_K_M-00004-of-00005.gguf +3 -0
  39. WizardLM-2-8x22B.Q4_K_M-00005-of-00005.gguf +3 -0
  40. WizardLM-2-8x22B.Q4_K_S-00001-of-00005.gguf +3 -0
  41. WizardLM-2-8x22B.Q4_K_S-00002-of-00005.gguf +3 -0
  42. WizardLM-2-8x22B.Q4_K_S-00003-of-00005.gguf +3 -0
  43. WizardLM-2-8x22B.Q4_K_S-00004-of-00005.gguf +3 -0
  44. WizardLM-2-8x22B.Q4_K_S-00005-of-00005.gguf +3 -0
  45. WizardLM-2-8x22B.Q5_K_M-00001-of-00005.gguf +3 -0
  46. WizardLM-2-8x22B.Q5_K_M-00002-of-00005.gguf +3 -0
  47. WizardLM-2-8x22B.Q5_K_M-00003-of-00005.gguf +3 -0
  48. WizardLM-2-8x22B.Q5_K_M-00004-of-00005.gguf +3 -0
  49. WizardLM-2-8x22B.Q5_K_M-00005-of-00005.gguf +3 -0
  50. WizardLM-2-8x22B.Q5_K_S-00001-of-00005.gguf +3 -0
.gitattributes ADDED
@@ -0,0 +1,97 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ *.7z filter=lfs diff=lfs merge=lfs -text
2
+ *.arrow filter=lfs diff=lfs merge=lfs -text
3
+ *.bin filter=lfs diff=lfs merge=lfs -text
4
+ *.bz2 filter=lfs diff=lfs merge=lfs -text
5
+ *.ckpt filter=lfs diff=lfs merge=lfs -text
6
+ *.ftz filter=lfs diff=lfs merge=lfs -text
7
+ *.gz filter=lfs diff=lfs merge=lfs -text
8
+ *.h5 filter=lfs diff=lfs merge=lfs -text
9
+ *.joblib filter=lfs diff=lfs merge=lfs -text
10
+ *.lfs.* filter=lfs diff=lfs merge=lfs -text
11
+ *.mlmodel filter=lfs diff=lfs merge=lfs -text
12
+ *.model filter=lfs diff=lfs merge=lfs -text
13
+ *.msgpack filter=lfs diff=lfs merge=lfs -text
14
+ *.npy filter=lfs diff=lfs merge=lfs -text
15
+ *.npz filter=lfs diff=lfs merge=lfs -text
16
+ *.onnx filter=lfs diff=lfs merge=lfs -text
17
+ *.ot filter=lfs diff=lfs merge=lfs -text
18
+ *.parquet filter=lfs diff=lfs merge=lfs -text
19
+ *.pb filter=lfs diff=lfs merge=lfs -text
20
+ *.pickle filter=lfs diff=lfs merge=lfs -text
21
+ *.pkl filter=lfs diff=lfs merge=lfs -text
22
+ *.pt filter=lfs diff=lfs merge=lfs -text
23
+ *.pth filter=lfs diff=lfs merge=lfs -text
24
+ *.rar filter=lfs diff=lfs merge=lfs -text
25
+ *.safetensors filter=lfs diff=lfs merge=lfs -text
26
+ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
27
+ *.tar.* filter=lfs diff=lfs merge=lfs -text
28
+ *.tar filter=lfs diff=lfs merge=lfs -text
29
+ *.tflite filter=lfs diff=lfs merge=lfs -text
30
+ *.tgz filter=lfs diff=lfs merge=lfs -text
31
+ *.wasm filter=lfs diff=lfs merge=lfs -text
32
+ *.xz filter=lfs diff=lfs merge=lfs -text
33
+ *.zip filter=lfs diff=lfs merge=lfs -text
34
+ *.zst filter=lfs diff=lfs merge=lfs -text
35
+ *tfevents* filter=lfs diff=lfs merge=lfs -text
36
+ WizardLM-2-8x22B.Q2_K-00001-of-00005.gguf filter=lfs diff=lfs merge=lfs -text
37
+ WizardLM-2-8x22B.Q2_K-00002-of-00005.gguf filter=lfs diff=lfs merge=lfs -text
38
+ WizardLM-2-8x22B.Q2_K-00003-of-00005.gguf filter=lfs diff=lfs merge=lfs -text
39
+ WizardLM-2-8x22B.Q2_K-00004-of-00005.gguf filter=lfs diff=lfs merge=lfs -text
40
+ WizardLM-2-8x22B.Q2_K-00005-of-00005.gguf filter=lfs diff=lfs merge=lfs -text
41
+ WizardLM-2-8x22B.IQ3_XS-00001-of-00005.gguf filter=lfs diff=lfs merge=lfs -text
42
+ WizardLM-2-8x22B.IQ3_XS-00002-of-00005.gguf filter=lfs diff=lfs merge=lfs -text
43
+ WizardLM-2-8x22B.IQ3_XS-00003-of-00005.gguf filter=lfs diff=lfs merge=lfs -text
44
+ WizardLM-2-8x22B.IQ3_XS-00004-of-00005.gguf filter=lfs diff=lfs merge=lfs -text
45
+ WizardLM-2-8x22B.IQ3_XS-00005-of-00005.gguf filter=lfs diff=lfs merge=lfs -text
46
+ WizardLM-2-8x22B.Q3_K_S-00001-of-00005.gguf filter=lfs diff=lfs merge=lfs -text
47
+ WizardLM-2-8x22B.Q3_K_S-00002-of-00005.gguf filter=lfs diff=lfs merge=lfs -text
48
+ WizardLM-2-8x22B.Q3_K_S-00003-of-00005.gguf filter=lfs diff=lfs merge=lfs -text
49
+ WizardLM-2-8x22B.Q3_K_S-00004-of-00005.gguf filter=lfs diff=lfs merge=lfs -text
50
+ WizardLM-2-8x22B.Q3_K_S-00005-of-00005.gguf filter=lfs diff=lfs merge=lfs -text
51
+ WizardLM-2-8x22B.Q3_K_L-00001-of-00005.gguf filter=lfs diff=lfs merge=lfs -text
52
+ WizardLM-2-8x22B.Q3_K_L-00002-of-00005.gguf filter=lfs diff=lfs merge=lfs -text
53
+ WizardLM-2-8x22B.Q3_K_L-00003-of-00005.gguf filter=lfs diff=lfs merge=lfs -text
54
+ WizardLM-2-8x22B.Q3_K_L-00004-of-00005.gguf filter=lfs diff=lfs merge=lfs -text
55
+ WizardLM-2-8x22B.Q3_K_L-00005-of-00005.gguf filter=lfs diff=lfs merge=lfs -text
56
+ WizardLM-2-8x22B.Q3_K_M-00001-of-00005.gguf filter=lfs diff=lfs merge=lfs -text
57
+ WizardLM-2-8x22B.Q3_K_M-00002-of-00005.gguf filter=lfs diff=lfs merge=lfs -text
58
+ WizardLM-2-8x22B.Q3_K_M-00003-of-00005.gguf filter=lfs diff=lfs merge=lfs -text
59
+ WizardLM-2-8x22B.Q3_K_M-00004-of-00005.gguf filter=lfs diff=lfs merge=lfs -text
60
+ WizardLM-2-8x22B.Q3_K_M-00005-of-00005.gguf filter=lfs diff=lfs merge=lfs -text
61
+ WizardLM-2-8x22B.Q4_K_S-00001-of-00005.gguf filter=lfs diff=lfs merge=lfs -text
62
+ WizardLM-2-8x22B.Q4_K_S-00002-of-00005.gguf filter=lfs diff=lfs merge=lfs -text
63
+ WizardLM-2-8x22B.Q4_K_S-00003-of-00005.gguf filter=lfs diff=lfs merge=lfs -text
64
+ WizardLM-2-8x22B.Q4_K_S-00004-of-00005.gguf filter=lfs diff=lfs merge=lfs -text
65
+ WizardLM-2-8x22B.Q4_K_S-00005-of-00005.gguf filter=lfs diff=lfs merge=lfs -text
66
+ WizardLM-2-8x22B.IQ4_XS-00001-of-00005.gguf filter=lfs diff=lfs merge=lfs -text
67
+ WizardLM-2-8x22B.IQ4_XS-00002-of-00005.gguf filter=lfs diff=lfs merge=lfs -text
68
+ WizardLM-2-8x22B.IQ4_XS-00003-of-00005.gguf filter=lfs diff=lfs merge=lfs -text
69
+ WizardLM-2-8x22B.IQ4_XS-00004-of-00005.gguf filter=lfs diff=lfs merge=lfs -text
70
+ WizardLM-2-8x22B.IQ4_XS-00005-of-00005.gguf filter=lfs diff=lfs merge=lfs -text
71
+ WizardLM-2-8x22B.Q4_K_M-00001-of-00005.gguf filter=lfs diff=lfs merge=lfs -text
72
+ WizardLM-2-8x22B.Q4_K_M-00002-of-00005.gguf filter=lfs diff=lfs merge=lfs -text
73
+ WizardLM-2-8x22B.Q4_K_M-00003-of-00005.gguf filter=lfs diff=lfs merge=lfs -text
74
+ WizardLM-2-8x22B.Q4_K_M-00004-of-00005.gguf filter=lfs diff=lfs merge=lfs -text
75
+ WizardLM-2-8x22B.Q4_K_M-00005-of-00005.gguf filter=lfs diff=lfs merge=lfs -text
76
+ WizardLM-2-8x22B.Q5_K_M-00001-of-00005.gguf filter=lfs diff=lfs merge=lfs -text
77
+ WizardLM-2-8x22B.Q5_K_M-00002-of-00005.gguf filter=lfs diff=lfs merge=lfs -text
78
+ WizardLM-2-8x22B.Q5_K_M-00003-of-00005.gguf filter=lfs diff=lfs merge=lfs -text
79
+ WizardLM-2-8x22B.Q5_K_M-00004-of-00005.gguf filter=lfs diff=lfs merge=lfs -text
80
+ WizardLM-2-8x22B.Q5_K_M-00005-of-00005.gguf filter=lfs diff=lfs merge=lfs -text
81
+ WizardLM-2-8x22B.Q5_K_S-00001-of-00005.gguf filter=lfs diff=lfs merge=lfs -text
82
+ WizardLM-2-8x22B.Q5_K_S-00002-of-00005.gguf filter=lfs diff=lfs merge=lfs -text
83
+ WizardLM-2-8x22B.Q5_K_S-00003-of-00005.gguf filter=lfs diff=lfs merge=lfs -text
84
+ WizardLM-2-8x22B.Q5_K_S-00004-of-00005.gguf filter=lfs diff=lfs merge=lfs -text
85
+ WizardLM-2-8x22B.Q5_K_S-00005-of-00005.gguf filter=lfs diff=lfs merge=lfs -text
86
+ WizardLM-2-8x22B.Q6_K-00001-of-00005.gguf filter=lfs diff=lfs merge=lfs -text
87
+ WizardLM-2-8x22B.Q6_K-00002-of-00005.gguf filter=lfs diff=lfs merge=lfs -text
88
+ WizardLM-2-8x22B.Q6_K-00003-of-00005.gguf filter=lfs diff=lfs merge=lfs -text
89
+ WizardLM-2-8x22B.Q6_K-00004-of-00005.gguf filter=lfs diff=lfs merge=lfs -text
90
+ WizardLM-2-8x22B.Q6_K-00005-of-00005.gguf filter=lfs diff=lfs merge=lfs -text
91
+ WizardLM-2-8x22B.Q8_0-00001-of-00005.gguf filter=lfs diff=lfs merge=lfs -text
92
+ WizardLM-2-8x22B.Q8_0-00002-of-00005.gguf filter=lfs diff=lfs merge=lfs -text
93
+ WizardLM-2-8x22B.Q8_0-00003-of-00005.gguf filter=lfs diff=lfs merge=lfs -text
94
+ WizardLM-2-8x22B.Q8_0-00004-of-00005.gguf filter=lfs diff=lfs merge=lfs -text
95
+ WizardLM-2-8x22B.Q8_0-00005-of-00005.gguf filter=lfs diff=lfs merge=lfs -text
96
+ WizardLM-2-8x22B.IQ1_M.gguf filter=lfs diff=lfs merge=lfs -text
97
+ WizardLM-2-8x22B.IQ1_S.gguf filter=lfs diff=lfs merge=lfs -text
README.md ADDED
@@ -0,0 +1,74 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ tags:
3
+ - quantized
4
+ - 2-bit
5
+ - 3-bit
6
+ - 4-bit
7
+ - 5-bit
8
+ - 6-bit
9
+ - 8-bit
10
+ - GGUF
11
+ - transformers
12
+ - safetensors
13
+ - mistral
14
+ - text-generation
15
+ - arxiv:2304.12244
16
+ - arxiv:2306.08568
17
+ - arxiv:2308.09583
18
+ - license:apache-2.0
19
+ - autotrain_compatible
20
+ - endpoints_compatible
21
+ - text-generation-inference
22
+ - region:us
23
+ - text-generation
24
+ model_name: WizardLM-2-8x22B-GGUF
25
+ base_model: microsoft/WizardLM-2-8x22B
26
+ inference: false
27
+ model_creator: microsoft
28
+ pipeline_tag: text-generation
29
+ quantized_by: MaziyarPanahi
30
+ ---
31
+ # [MaziyarPanahi/WizardLM-2-8x22B-GGUF](https://huggingface.co/MaziyarPanahi/WizardLM-2-8x22B-GGUF)
32
+ - Model creator: [microsoft](https://huggingface.co/microsoft)
33
+ - Original model: [microsoft/WizardLM-2-8x22B](https://huggingface.co/microsoft/WizardLM-2-8x22B)
34
+
35
+ ## Description
36
+ [MaziyarPanahi/WizardLM-2-8x22B-GGUF](https://huggingface.co/MaziyarPanahi/WizardLM-2-8x22B-GGUF) contains GGUF format model files for [microsoft/WizardLM-2-8x22B](https://huggingface.co/microsoft/WizardLM-2-8x22B).
37
+
38
+ ## How to download
39
+ You can download only the quants you need instead of cloning the entire repository as follows:
40
+
41
+
42
+ ```
43
+ huggingface-cli download MaziyarPanahi/WizardLM-2-8x22B-GGUF --local-dir . --include '*Q2_K*gguf'
44
+ ```
45
+
46
+ On Windows:
47
+
48
+ ```sh
49
+ huggingface-cli download MaziyarPanahi/WizardLM-2-8x22B-GGUF --local-dir . --include *Q4_K_S*gguf
50
+ ```
51
+
52
+ ## Load sharded model
53
+
54
+ `llama_load_model_from_file` will detect the number of files and will load additional tensors from the rest of files.
55
+
56
+ ```sh
57
+ llama.cpp/main -m WizardLM-2-8x22B.Q2_K-00001-of-00005.gguf -p "Building a website can be done in 10 simple steps:\nStep 1:" -n 1024 -e
58
+ ```
59
+
60
+ ## Prompt template
61
+
62
+ ```
63
+ {system_prompt}
64
+ USER: {prompt}
65
+ ASSISTANT: </s>
66
+ ```
67
+
68
+ or
69
+
70
+ ```
71
+ A chat between a curious user and an artificial intelligence assistant. The assistant gives helpful,
72
+ detailed, and polite answers to the user's questions. USER: Hi ASSISTANT: Hello.</s>
73
+ USER: {prompt} ASSISTANT: </s>......
74
+ ```
WizardLM-2-8x22B.IQ1_M.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:c3d218eb61333cd15a5fe8ca5389ae325a05b6cf1f54a9cb3a0ccf51106aa4b8
3
+ size 32732397696
WizardLM-2-8x22B.IQ1_S.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:8f898f5df6c1ebf334bbc8187bc3cd6789881470a1000ce846eccca5b8e59313
3
+ size 29644079232
WizardLM-2-8x22B.IQ3_XS-00001-of-00005.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:507606f84c52f2f3e73985fa7717f491a9a6a1328d4650d023ebecaef8033614
3
+ size 13520020416
WizardLM-2-8x22B.IQ3_XS-00002-of-00005.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:8e566a04d762cddc4579b844e425c614b46e2266756e7255e99d03bd93c3144e
3
+ size 13483630784
WizardLM-2-8x22B.IQ3_XS-00003-of-00005.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:fafb55b1e73da0c6ababd35b4086578f0c29b1a04b5492ad3e3475660f85ddab
3
+ size 12270264448
WizardLM-2-8x22B.IQ3_XS-00004-of-00005.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:e421a93a1fcbd9ff769cc5eed1e74b8c3ab281ecdc7977f915c6929ffa9859b7
3
+ size 13339615392
WizardLM-2-8x22B.IQ3_XS-00005-of-00005.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:37afb9affd4192da35e6d4e4879a79571457b1d2b2d358f5ab269978c1bc506c
3
+ size 5614673216
WizardLM-2-8x22B.IQ4_XS-00001-of-00005.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:f81c943d1c259a6d5309f29d914ddc2aec28b4ca5faf0dbd782106f894533f9f
3
+ size 18016850880
WizardLM-2-8x22B.IQ4_XS-00002-of-00005.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:8ea1881a0b42f37767af47c1d13b9eb0efe510853a731b678feabff97c015719
3
+ size 17847247040
WizardLM-2-8x22B.IQ4_XS-00003-of-00005.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:52ae1b3b1b75dabaf96484ec1e0141f0ce65225f1b98a73d2261a50d6cae3f15
3
+ size 16187875456
WizardLM-2-8x22B.IQ4_XS-00004-of-00005.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:ce912e4e6c0948169b3b192268cbdb04af0c1bd8f3790bfcb9cd2fc39d4f0291
3
+ size 17432699040
WizardLM-2-8x22B.IQ4_XS-00005-of-00005.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:762ac686d247b8b7499ea8076c432df8f409807401819c47f7042be617ad8bce
3
+ size 6869523776
WizardLM-2-8x22B.Q2_K-00001-of-00005.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:906b9d1f7a91fe5e3acb123bd45cc032096bcd4214d5ae2f4a24185eaf980bf4
3
+ size 11843728320
WizardLM-2-8x22B.Q2_K-00002-of-00005.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:53a3fe6da0e7076c168ff7c94b454ec98d0c46c575d27ece5236be21c61af3f6
3
+ size 12267708608
WizardLM-2-8x22B.Q2_K-00003-of-00005.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:4d6434c42e263006300fddbd9f3b923a6cc690ddd350847579c65dc178ed0bea
3
+ size 11188527232
WizardLM-2-8x22B.Q2_K-00004-of-00005.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:af766c6d298310821a4ab132bc8e6a21817023ca428ffa2bb6a54b9d2da45600
3
+ size 12016738464
WizardLM-2-8x22B.Q2_K-00005-of-00005.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:6a7fd3949c13b60d1369f86663cef9ca4e900037e9c621ab39db2790ec64b519
3
+ size 4786462016
WizardLM-2-8x22B.Q3_K_L-00001-of-00005.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:677254a8764e9cdc736e638e6634c960c3dbb305c99eb15963f5edb28b4f81f2
3
+ size 16530777024
WizardLM-2-8x22B.Q3_K_L-00002-of-00005.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:b3c810fd9c410f7d21effb1d75fb1255a3a06770a696755d8bdadd618f07bf58
3
+ size 17108099264
WizardLM-2-8x22B.Q3_K_L-00003-of-00005.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:5129af244ebaaecbf40f89cb6eede1b2035c4f9a8f1df066ed97413e87fb4eec
3
+ size 15552635008
WizardLM-2-8x22B.Q3_K_L-00004-of-00005.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:ecef6b5d484ae03d0684ec579fb086e3864cf2b50805c85dd3a6dbffb69fd8fe
3
+ size 16775340192
WizardLM-2-8x22B.Q3_K_L-00005-of-00005.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:af2364a5dc9bde78cfccf90063df9b01658410629124ceadb40eef2edb9f628b
3
+ size 6612950336
WizardLM-2-8x22B.Q3_K_M-00001-of-00005.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:6b4132b1115f0fa12451c9049c24c0f3d1ab3348f70fa4609a94d193a6d58da1
3
+ size 15650661312
WizardLM-2-8x22B.Q3_K_M-00002-of-00005.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:001d7eb7125217a694781fecf9ca2cb49c655fb5b0bd124a4025b198dd140bae
3
+ size 15916261568
WizardLM-2-8x22B.Q3_K_M-00003-of-00005.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:577ea20c1fcc89ee42e5fabdd372dc7974857bf0eaedfab111ca94c6ba0fc607
3
+ size 14480924800
WizardLM-2-8x22B.Q3_K_M-00004-of-00005.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:1a96a5ad47fa40eebd46723a2db791e5834430bbb1a25014839028fa64db87bf
3
+ size 15583502496
WizardLM-2-8x22B.Q3_K_M-00005-of-00005.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:f79c807121753e7c4d8312f02330c5315a37e47ac1e1feb6a64f395416fb6af1
3
+ size 6158294336
WizardLM-2-8x22B.Q3_K_S-00001-of-00005.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:02c67f83658f8039c146642741936c7a1ad18d786af0364e7486f616c45508fc
3
+ size 13958259648
WizardLM-2-8x22B.Q3_K_S-00002-of-00005.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:4578f7ef6150c93b7347bdb77c6f2c3f839bec145ec9022bccea8900d8e3caf2
3
+ size 14525849792
WizardLM-2-8x22B.Q3_K_S-00003-of-00005.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:bbf516d44516ac5f1478f48dfb37c45ebd91940111b32af93edbcf629441e8a5
3
+ size 13197467776
WizardLM-2-8x22B.Q3_K_S-00004-of-00005.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:5f4433ced22748b00fabffe18ef41b93aa164214029603286ecc55ad7ef6f8e1
3
+ size 14193090720
WizardLM-2-8x22B.Q3_K_S-00005-of-00005.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:13b984ca43dcbf4d0a873d7658400060ab00629cc784716142819dbb045da01c
3
+ size 5623520576
WizardLM-2-8x22B.Q4_K_M-00001-of-00005.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:1ff155aaade7d6d9c208e5ad3933cbc02974c30a6742ee3404f7b51e617a4f3e
3
+ size 19982390208
WizardLM-2-8x22B.Q4_K_M-00002-of-00005.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:cda9d26caeeb40eb916e04b4ef5efc0117b90ffe63d5f8fe997262fb04b4b333
3
+ size 19699687616
WizardLM-2-8x22B.Q4_K_M-00003-of-00005.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:2765d9a73e1d558781fcd77b908d3a26ce92772b02a2cf38ac0717f95f1ab58f
3
+ size 17938473088
WizardLM-2-8x22B.Q4_K_M-00004-of-00005.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:ee19ae577798150ee11f6cffe220f9257e4658dcfb6eec6b430aaf97aaa42ba6
3
+ size 19675209888
WizardLM-2-8x22B.Q4_K_M-00005-of-00005.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:4bef8ea8f78d69ad3b9b13f59ee6b69234732670e31580b201f56e4091e1419f
3
+ size 8290999616
WizardLM-2-8x22B.Q4_K_S-00001-of-00005.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:92456ec422ddf695a19c990df3ac4186f0b12cc757504e3970d200d449044e3a
3
+ size 18818470848
WizardLM-2-8x22B.Q4_K_S-00002-of-00005.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:0c5c0adcd16f8254086b04498820138236615e71c7a50b01a78523530a2da260
3
+ size 18869215424
WizardLM-2-8x22B.Q4_K_S-00003-of-00005.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:7ab35ad5d7f7e4e8cd8cfafb2291b7ff5e8baa595cfcfaca209fa4dfc7c022bc
3
+ size 17108000896
WizardLM-2-8x22B.Q4_K_S-00004-of-00005.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:6e84aa55e44859ffb4da18f320bdef3b705ede320ad5118ed5d3bc3fd8717d39
3
+ size 18429501600
WizardLM-2-8x22B.Q4_K_S-00005-of-00005.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:966d15e0c80a8305614d44b0337fbad8c4253bd8bf0973161d39ae2695769110
3
+ size 7252909376
WizardLM-2-8x22B.Q5_K_M-00001-of-00005.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:8fa60ba9d5be2dae5875904f22716eb3cf7fdf99a9871304e324e403369cfc58
3
+ size 22987543488
WizardLM-2-8x22B.Q5_K_M-00002-of-00005.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:0acd9445e3681da199b3243ba9467952106a5d599af0226626c63d2431da6129
3
+ size 23384907968
WizardLM-2-8x22B.Q5_K_M-00003-of-00005.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:29febc013ad3af2c7201bb81433f596e25fa75c060821e20edb65cb1eacb708f
3
+ size 21216321664
WizardLM-2-8x22B.Q5_K_M-00004-of-00005.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:df9a71ae9a4e57454c1812a51171a5729dee28f0ff1f657cf10aaf79ecd094e9
3
+ size 23058440352
WizardLM-2-8x22B.Q5_K_M-00005-of-00005.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:6706cb5d3134b1722aa39835debbb666f3183ec26b0bb09f70448c796adfb73b
3
+ size 9321225536
WizardLM-2-8x22B.Q5_K_S-00001-of-00005.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:72fa0b4a2b0b09a62e751473c4fa04f0e60fc71ca7b3dbc127cd7dd658375ec0
3
+ size 22024950720