Hugging Face's logo

bartowski
/
Qwen1.5-7B-Chat-GGUF

Text Generation
GGUF
English
chat
Model card Files Community
bartowski commited on
Commit
c4b7515
•
1 Parent(s): 249e3b8

Llamacpp quants

Browse files
Files changed (14) hide show
  1. .gitattributes +12 -0
  2. Qwen1.5-7B-Chat-Q2_K.gguf +3 -0
  3. Qwen1.5-7B-Chat-Q3_K_L.gguf +3 -0
  4. Qwen1.5-7B-Chat-Q3_K_M.gguf +3 -0
  5. Qwen1.5-7B-Chat-Q3_K_S.gguf +3 -0
  6. Qwen1.5-7B-Chat-Q4_0.gguf +3 -0
  7. Qwen1.5-7B-Chat-Q4_K_M.gguf +3 -0
  8. Qwen1.5-7B-Chat-Q4_K_S.gguf +3 -0
  9. Qwen1.5-7B-Chat-Q5_0.gguf +3 -0
  10. Qwen1.5-7B-Chat-Q5_K_M.gguf +3 -0
  11. Qwen1.5-7B-Chat-Q5_K_S.gguf +3 -0
  12. Qwen1.5-7B-Chat-Q6_K.gguf +3 -0
  13. Qwen1.5-7B-Chat-Q8_0.gguf +3 -0
  14. README.md +37 -0
.gitattributes CHANGED
@@ -33,3 +33,15 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
 
 
 
 
 
 
 
 
 
 
 
 
 
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
36
+ Qwen1.5-7B-Chat-Q2_K.gguf filter=lfs diff=lfs merge=lfs -text
37
+ Qwen1.5-7B-Chat-Q3_K_L.gguf filter=lfs diff=lfs merge=lfs -text
38
+ Qwen1.5-7B-Chat-Q3_K_M.gguf filter=lfs diff=lfs merge=lfs -text
39
+ Qwen1.5-7B-Chat-Q3_K_S.gguf filter=lfs diff=lfs merge=lfs -text
40
+ Qwen1.5-7B-Chat-Q4_0.gguf filter=lfs diff=lfs merge=lfs -text
41
+ Qwen1.5-7B-Chat-Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text
42
+ Qwen1.5-7B-Chat-Q4_K_S.gguf filter=lfs diff=lfs merge=lfs -text
43
+ Qwen1.5-7B-Chat-Q5_0.gguf filter=lfs diff=lfs merge=lfs -text
44
+ Qwen1.5-7B-Chat-Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text
45
+ Qwen1.5-7B-Chat-Q5_K_S.gguf filter=lfs diff=lfs merge=lfs -text
46
+ Qwen1.5-7B-Chat-Q6_K.gguf filter=lfs diff=lfs merge=lfs -text
47
+ Qwen1.5-7B-Chat-Q8_0.gguf filter=lfs diff=lfs merge=lfs -text
Qwen1.5-7B-Chat-Q2_K.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:85f00a2dd964b173f2addcd13713673b65bc6c1d2232dac2a76bb7bdb45a18a5
3
+ size 3103825312
Qwen1.5-7B-Chat-Q3_K_L.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:b2dea08afd336a3287dd0a4c100e586dbe3e7a21566d45ab50952f6a42515aec
3
+ size 4217965984
Qwen1.5-7B-Chat-Q3_K_M.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:846ed4c845c24fa09207913c147b071809a7b0551f7f1d9c54381da42a41c51c
3
+ size 3918859680
Qwen1.5-7B-Chat-Q3_K_S.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:a8be3dc3102d26b39db2852386d805428b668be2b4c1e6ec26f457219cba4e3e
3
+ size 3569159584
Qwen1.5-7B-Chat-Q4_0.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:f887191a62588136ea88c2c7aac2c518659d163f441601b2fe8591af7da8b020
3
+ size 4511907232
Qwen1.5-7B-Chat-Q4_K_M.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:2d9934f298591626fdaa27c100685ba940b5a03aff5d639736880b234ec03cd9
3
+ size 4767104416
Qwen1.5-7B-Chat-Q4_K_S.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:560a297e8365905150362e5df009f4e97520abd881ec0a5ff6604bfd4d0d3e90
3
+ size 4542840224
Qwen1.5-7B-Chat-Q5_0.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:8d39cfe41b336d96963b0e3999443cd329d893db4aab32eafada089740d620fb
3
+ size 5399199136
Qwen1.5-7B-Chat-Q5_K_M.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:14820d77b036c97fe263b6fe981ce39750d97eab0e6c274f40a5bbce7be0775c
3
+ size 5530664352
Qwen1.5-7B-Chat-Q5_K_S.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:8bede671e24eb12e305d7116bccb4267be7ee35819526e7dff7aa598380cfd5b
3
+ size 5399199136
Qwen1.5-7B-Chat-Q6_K.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:da31621de3e864bbcb56f4c7d9bcfbf59563a64da8b44cfbdc97905f130e9596
3
+ size 6341946784
Qwen1.5-7B-Chat-Q8_0.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:5fb7bc8a8870d5081dc326809c57c62c05f27c7be17541ef449373e60e4bca6a
3
+ size 8211795360
README.md ADDED
@@ -0,0 +1,37 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: other
3
+ license_name: tongyi-qianwen
4
+ license_link: >-
5
+ https://huggingface.co/Qwen/Qwen1.5-7B-Chat/blob/main/LICENSE
6
+ language:
7
+ - en
8
+ pipeline_tag: text-generation
9
+ tags:
10
+ - chat
11
+ quantized_by: bartowski
12
+ ---
13
+
14
+ ## Llamacpp Quantizations of Qwen1.5-7B-Chat
15
+
16
+ Using <a href="https://github.com/ggerganov/llama.cpp/">llama.cpp</a> release <a href="https://github.com/ggerganov/llama.cpp/releases/tag/b2440">b2440</a> for quantization.
17
+
18
+ Original model: https://huggingface.co/Qwen/Qwen1.5-7B-Chat
19
+
20
+ Download a file (not the whole branch) from below:
21
+
22
+ | Filename | Quant type | File Size | Description |
23
+ | -------- | ---------- | --------- | ----------- |
24
+ | [Qwen1.5-7B-Chat-Q8_0.gguf](https://huggingface.co/bartowski/Qwen1.5-7B-Chat-GGUF/blob/main/Qwen1.5-7B-Chat-Q8_0.gguf) | Q8_0 | 8.21GB | Extremely high quality, generally unneeded but max available quant. |
25
+ | [Qwen1.5-7B-Chat-Q6_K.gguf](https://huggingface.co/bartowski/Qwen1.5-7B-Chat-GGUF/blob/main/Qwen1.5-7B-Chat-Q6_K.gguf) | Q6_K | 6.34GB | Very high quality, near perfect, *recommended*. |
26
+ | [Qwen1.5-7B-Chat-Q5_K_M.gguf](https://huggingface.co/bartowski/Qwen1.5-7B-Chat-GGUF/blob/main/Qwen1.5-7B-Chat-Q5_K_M.gguf) | Q5_K_M | 5.53GB | High quality, very usable. |
27
+ | [Qwen1.5-7B-Chat-Q5_K_S.gguf](https://huggingface.co/bartowski/Qwen1.5-7B-Chat-GGUF/blob/main/Qwen1.5-7B-Chat-Q5_K_S.gguf) | Q5_K_S | 5.39GB | High quality, very usable. |
28
+ | [Qwen1.5-7B-Chat-Q5_0.gguf](https://huggingface.co/bartowski/Qwen1.5-7B-Chat-GGUF/blob/main/Qwen1.5-7B-Chat-Q5_0.gguf) | Q5_0 | 5.39GB | High quality, older format, generally not recommended. |
29
+ | [Qwen1.5-7B-Chat-Q4_K_M.gguf](https://huggingface.co/bartowski/Qwen1.5-7B-Chat-GGUF/blob/main/Qwen1.5-7B-Chat-Q4_K_M.gguf) | Q4_K_M | 4.76GB | Good quality, similar to 4.25 bpw. |
30
+ | [Qwen1.5-7B-Chat-Q4_K_S.gguf](https://huggingface.co/bartowski/Qwen1.5-7B-Chat-GGUF/blob/main/Qwen1.5-7B-Chat-Q4_K_S.gguf) | Q4_K_S | 4.54GB | Slightly lower quality with small space savings. |
31
+ | [Qwen1.5-7B-Chat-Q4_0.gguf](https://huggingface.co/bartowski/Qwen1.5-7B-Chat-GGUF/blob/main/Qwen1.5-7B-Chat-Q4_0.gguf) | Q4_0 | 4.51GB | Decent quality, older format, generally not recommended. |
32
+ | [Qwen1.5-7B-Chat-Q3_K_L.gguf](https://huggingface.co/bartowski/Qwen1.5-7B-Chat-GGUF/blob/main/Qwen1.5-7B-Chat-Q3_K_L.gguf) | Q3_K_L | 4.21GB | Lower quality but usable, good for low RAM availability. |
33
+ | [Qwen1.5-7B-Chat-Q3_K_M.gguf](https://huggingface.co/bartowski/Qwen1.5-7B-Chat-GGUF/blob/main/Qwen1.5-7B-Chat-Q3_K_M.gguf) | Q3_K_M | 3.91GB | Even lower quality. |
34
+ | [Qwen1.5-7B-Chat-Q3_K_S.gguf](https://huggingface.co/bartowski/Qwen1.5-7B-Chat-GGUF/blob/main/Qwen1.5-7B-Chat-Q3_K_S.gguf) | Q3_K_S | 3.56GB | Low quality, not recommended. |
35
+ | [Qwen1.5-7B-Chat-Q2_K.gguf](https://huggingface.co/bartowski/Qwen1.5-7B-Chat-GGUF/blob/main/Qwen1.5-7B-Chat-Q2_K.gguf) | Q2_K | 3.10GB | Extremely low quality, *not* recommended.
36
+
37
+ Want to support my work? Visit my ko-fi page here: https://ko-fi.com/bartowski