Hugging Face's logo

bartowski
/
blossom-v5-14b-GGUF

Text Generation
GGUF
Chinese
English
Model card Files Community
bartowski commited on
Commit
8600181
•
1 Parent(s): 3005406

Llamacpp quants

Browse files
Files changed (18) hide show
  1. .gitattributes +16 -0
  2. README.md +42 -0
  3. blossom-v5-14b-IQ3_M.gguf +3 -0
  4. blossom-v5-14b-IQ3_S.gguf +3 -0
  5. blossom-v5-14b-IQ4_NL.gguf +3 -0
  6. blossom-v5-14b-IQ4_XS.gguf +3 -0
  7. blossom-v5-14b-Q2_K.gguf +3 -0
  8. blossom-v5-14b-Q3_K_L.gguf +3 -0
  9. blossom-v5-14b-Q3_K_M.gguf +3 -0
  10. blossom-v5-14b-Q3_K_S.gguf +3 -0
  11. blossom-v5-14b-Q4_0.gguf +3 -0
  12. blossom-v5-14b-Q4_K_M.gguf +3 -0
  13. blossom-v5-14b-Q4_K_S.gguf +3 -0
  14. blossom-v5-14b-Q5_0.gguf +3 -0
  15. blossom-v5-14b-Q5_K_M.gguf +3 -0
  16. blossom-v5-14b-Q5_K_S.gguf +3 -0
  17. blossom-v5-14b-Q6_K.gguf +3 -0
  18. blossom-v5-14b-Q8_0.gguf +3 -0
.gitattributes CHANGED
@@ -33,3 +33,19 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
36
+ blossom-v5-14b-IQ3_M.gguf filter=lfs diff=lfs merge=lfs -text
37
+ blossom-v5-14b-IQ3_S.gguf filter=lfs diff=lfs merge=lfs -text
38
+ blossom-v5-14b-IQ4_NL.gguf filter=lfs diff=lfs merge=lfs -text
39
+ blossom-v5-14b-IQ4_XS.gguf filter=lfs diff=lfs merge=lfs -text
40
+ blossom-v5-14b-Q2_K.gguf filter=lfs diff=lfs merge=lfs -text
41
+ blossom-v5-14b-Q3_K_L.gguf filter=lfs diff=lfs merge=lfs -text
42
+ blossom-v5-14b-Q3_K_M.gguf filter=lfs diff=lfs merge=lfs -text
43
+ blossom-v5-14b-Q3_K_S.gguf filter=lfs diff=lfs merge=lfs -text
44
+ blossom-v5-14b-Q4_0.gguf filter=lfs diff=lfs merge=lfs -text
45
+ blossom-v5-14b-Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text
46
+ blossom-v5-14b-Q4_K_S.gguf filter=lfs diff=lfs merge=lfs -text
47
+ blossom-v5-14b-Q5_0.gguf filter=lfs diff=lfs merge=lfs -text
48
+ blossom-v5-14b-Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text
49
+ blossom-v5-14b-Q5_K_S.gguf filter=lfs diff=lfs merge=lfs -text
50
+ blossom-v5-14b-Q6_K.gguf filter=lfs diff=lfs merge=lfs -text
51
+ blossom-v5-14b-Q8_0.gguf filter=lfs diff=lfs merge=lfs -text
README.md ADDED
@@ -0,0 +1,42 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: apache-2.0
3
+ datasets:
4
+ - Azure99/blossom-chat-v3
5
+ - Azure99/blossom-math-v4
6
+ - Azure99/blossom-wizard-v3
7
+ - Azure99/blossom-orca-v3
8
+ language:
9
+ - zh
10
+ - en
11
+ quantized_by: bartowski
12
+ pipeline_tag: text-generation
13
+ ---
14
+
15
+ ## Llamacpp Quantizations of blossom-v5-14b
16
+
17
+ Using <a href="https://github.com/ggerganov/llama.cpp/">llama.cpp</a> release <a href="https://github.com/ggerganov/llama.cpp/releases/tag/b2440">b2440</a> for quantization.
18
+
19
+ Original model: https://huggingface.co/Azure99/blossom-v5-14b
20
+
21
+ Download a file (not the whole branch) from below:
22
+
23
+ | Filename | Quant type | File Size | Description |
24
+ | -------- | ---------- | --------- | ----------- |
25
+ | [blossom-v5-14b-Q8_0.gguf](https://huggingface.co/bartowski/blossom-v5-14b-GGUF/blob/main/blossom-v5-14b-Q8_0.gguf) | Q8_0 | 15.06GB | Extremely high quality, generally unneeded but max available quant. |
26
+ | [blossom-v5-14b-Q6_K.gguf](https://huggingface.co/bartowski/blossom-v5-14b-GGUF/blob/main/blossom-v5-14b-Q6_K.gguf) | Q6_K | 12.31GB | Very high quality, near perfect, *recommended*. |
27
+ | [blossom-v5-14b-Q5_K_M.gguf](https://huggingface.co/bartowski/blossom-v5-14b-GGUF/blob/main/blossom-v5-14b-Q5_K_M.gguf) | Q5_K_M | 10.53GB | High quality, very usable. |
28
+ | [blossom-v5-14b-Q5_K_S.gguf](https://huggingface.co/bartowski/blossom-v5-14b-GGUF/blob/main/blossom-v5-14b-Q5_K_S.gguf) | Q5_K_S | 10.02GB | High quality, very usable. |
29
+ | [blossom-v5-14b-Q5_0.gguf](https://huggingface.co/bartowski/blossom-v5-14b-GGUF/blob/main/blossom-v5-14b-Q5_0.gguf) | Q5_0 | 9.85GB | High quality, older format, generally not recommended. |
30
+ | [blossom-v5-14b-Q4_K_M.gguf](https://huggingface.co/bartowski/blossom-v5-14b-GGUF/blob/main/blossom-v5-14b-Q4_K_M.gguf) | Q4_K_M | 9.19GB | Good quality, similar to 4.25 bpw. |
31
+ | [blossom-v5-14b-Q4_K_S.gguf](https://huggingface.co/bartowski/blossom-v5-14b-GGUF/blob/main/blossom-v5-14b-Q4_K_S.gguf) | Q4_K_S | 8.56GB | Slightly lower quality with small space savings. |
32
+ | [blossom-v5-14b-IQ4_NL.gguf](https://huggingface.co/bartowski/blossom-v5-14b-GGUF/blob/main/blossom-v5-14b-IQ4_NL.gguf) | IQ4_NL | 8.24GB | Good quality, similar to Q4_K_S, new method of quanting, |
33
+ | [blossom-v5-14b-IQ4_XS.gguf](https://huggingface.co/bartowski/blossom-v5-14b-GGUF/blob/main/blossom-v5-14b-IQ4_XS.gguf) | IQ4_XS | 7.91GB | Decent quality, new method with similar performance to Q4. |
34
+ | [blossom-v5-14b-Q4_0.gguf](https://huggingface.co/bartowski/blossom-v5-14b-GGUF/blob/main/blossom-v5-14b-Q4_0.gguf) | Q4_0 | 8.17GB | Decent quality, older format, generally not recommended. |
35
+ | [blossom-v5-14b-IQ3_M.gguf](https://huggingface.co/bartowski/blossom-v5-14b-GGUF/blob/main/blossom-v5-14b-IQ3_M.gguf) | IQ3_M | 7.09GB | Medium-low quality, new method with decent performance. |
36
+ | [blossom-v5-14b-IQ3_S.gguf](https://huggingface.co/bartowski/blossom-v5-14b-GGUF/blob/main/blossom-v5-14b-IQ3_S.gguf) | IQ3_S | 6.77GB | Lower quality, new method with decent performance, recommended over Q3 quants. |
37
+ | [blossom-v5-14b-Q3_K_L.gguf](https://huggingface.co/bartowski/blossom-v5-14b-GGUF/blob/main/blossom-v5-14b-Q3_K_L.gguf) | Q3_K_L | 7.84GB | Lower quality but usable, good for low RAM availability. |
38
+ | [blossom-v5-14b-Q3_K_M.gguf](https://huggingface.co/bartowski/blossom-v5-14b-GGUF/blob/main/blossom-v5-14b-Q3_K_M.gguf) | Q3_K_M | 7.41GB | Even lower quality. |
39
+ | [blossom-v5-14b-Q3_K_S.gguf](https://huggingface.co/bartowski/blossom-v5-14b-GGUF/blob/main/blossom-v5-14b-Q3_K_S.gguf) | Q3_K_S | 6.77GB | Low quality, not recommended. |
40
+ | [blossom-v5-14b-Q2_K.gguf](https://huggingface.co/bartowski/blossom-v5-14b-GGUF/blob/main/blossom-v5-14b-Q2_K.gguf) | Q2_K | 5.91GB | Extremely low quality, *not* recommended.
41
+
42
+ Want to support my work? Visit my ko-fi page here: https://ko-fi.com/bartowski
blossom-v5-14b-IQ3_M.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:e811c9aa85fa2570ff98f2821af3ffe39b60de99a00b29a40c87878c1820eb7c
3
+ size 7096155328
blossom-v5-14b-IQ3_S.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:ba0212b9d681ede7b9a3ccac0df4441fcb558db5355bfdce78e7f0aed3d1e43d
3
+ size 6773800128
blossom-v5-14b-IQ4_NL.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:62814e610e025e02b3d703fd0fa938ab7539b885566e5fd0490964fede43f12a
3
+ size 8245062848
blossom-v5-14b-IQ4_XS.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:d330d580cdd04ea9b0d5c8812722cf3f1d2208ff5d4165653348f314d02bde88
3
+ size 7914351808
blossom-v5-14b-Q2_K.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:3aa091d37adb283aa6a00108b3ff5ffb10aa8e01d91a5c6d493103ce4458caae
3
+ size 5911981248
blossom-v5-14b-Q3_K_L.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:6ed0f0fc9c74a09ba0c02e3676c38de022d3e88934a1777322cc959c900c179e
3
+ size 7840398528
blossom-v5-14b-Q3_K_M.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:6a52a962d6c7eeec9c75c96e5b91c234efac9b576fcd1777216d96cfb82b5d8c
3
+ size 7418264768
blossom-v5-14b-Q3_K_S.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:1a05e226c3604bb03ccbf3fb556d8a8f482e91e4c3b57b9142f3fd021d27c83f
3
+ size 6773800128
blossom-v5-14b-Q4_0.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:1e8bfaac4c8be02889c23d5a915146a1e0b3772bb0abdf3aa99fff15f4645471
3
+ size 8179322048
blossom-v5-14b-Q4_K_M.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:6f34fb2ce40be8abc815cbce9c86fcf4cf1d135c7bbe3e41be77813c114d6a57
3
+ size 9191034048
blossom-v5-14b-Q4_K_S.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:be4ea7251e507cacb643e7638de8fd23ec3923fcaf1e5635d19da3414d31be7b
3
+ size 8564960448
blossom-v5-14b-Q5_0.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:f63209fbdd142afe2c058eb94d2635e0788cfe85c051f739774ee1bac2e57b01
3
+ size 9852783808
blossom-v5-14b-Q5_K_M.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:f7efcc251b7103a6e9d4c9d3d874a982728950784b869befe2e3afd8d8505d8f
3
+ size 10535996608
blossom-v5-14b-Q5_K_S.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:d3bab8a4e810c90beddb928e501d7119783a8648d41c4acd86493647deb451d2
3
+ size 10028092608
blossom-v5-14b-Q6_K.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:e45eae5527ea2fff7bc7ad1b1937a09e6cab29c2622094e18230000637495351
3
+ size 12310158528
blossom-v5-14b-Q8_0.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:ddba56a1bbf2941b3d9ad30dfd4e084534947c1b055e4205ddb7d3d3fd9c113e
3
+ size 15061728448