Hugging Face's logo

bartowski
/
Tess-2.0-Yi-34B-200K-GGUF

Text Generation
GGUF
Inference Endpoints
Model card Files Community
bartowski commited on
Commit
596f3c9
•
1 Parent(s): d13408b

Llamacpp quants

Browse files
Files changed (21) hide show
  1. .gitattributes +18 -0
  2. README.md +36 -0
  3. Tess-2.0-Yi-34B-200K-IQ3_M.gguf +3 -0
  4. Tess-2.0-Yi-34B-200K-IQ3_S.gguf +3 -0
  5. Tess-2.0-Yi-34B-200K-IQ4_NL.gguf +3 -0
  6. Tess-2.0-Yi-34B-200K-IQ4_XS.gguf +3 -0
  7. Tess-2.0-Yi-34B-200K-Q2_K.gguf +3 -0
  8. Tess-2.0-Yi-34B-200K-Q3_K_L.gguf +3 -0
  9. Tess-2.0-Yi-34B-200K-Q3_K_M.gguf +3 -0
  10. Tess-2.0-Yi-34B-200K-Q3_K_S.gguf +3 -0
  11. Tess-2.0-Yi-34B-200K-Q4_0.gguf +3 -0
  12. Tess-2.0-Yi-34B-200K-Q4_K_M.gguf +3 -0
  13. Tess-2.0-Yi-34B-200K-Q4_K_S.gguf +3 -0
  14. Tess-2.0-Yi-34B-200K-Q5_0.gguf +3 -0
  15. Tess-2.0-Yi-34B-200K-Q5_K_M.gguf +3 -0
  16. Tess-2.0-Yi-34B-200K-Q5_K_S.gguf +3 -0
  17. Tess-2.0-Yi-34B-200K-Q6_K.gguf +3 -0
  18. Tess-2.0-Yi-34B-200K-Q8_0.gguf +3 -0
  19. Tess-2.0-Yi-34B-200K-fp16.gguf/Tess-2.0-Yi-34B-200K-fp16_part_a +3 -0
  20. Tess-2.0-Yi-34B-200K-fp16.gguf/Tess-2.0-Yi-34B-200K-fp16_part_b +3 -0
  21. Tess-2.0-Yi-34B-200K-fp16.gguf/combine.sh +2 -0
.gitattributes CHANGED
@@ -33,3 +33,21 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
36
+ Tess-2.0-Yi-34B-200K-IQ3_M.gguf filter=lfs diff=lfs merge=lfs -text
37
+ Tess-2.0-Yi-34B-200K-IQ3_S.gguf filter=lfs diff=lfs merge=lfs -text
38
+ Tess-2.0-Yi-34B-200K-IQ4_NL.gguf filter=lfs diff=lfs merge=lfs -text
39
+ Tess-2.0-Yi-34B-200K-IQ4_XS.gguf filter=lfs diff=lfs merge=lfs -text
40
+ Tess-2.0-Yi-34B-200K-Q2_K.gguf filter=lfs diff=lfs merge=lfs -text
41
+ Tess-2.0-Yi-34B-200K-Q3_K_L.gguf filter=lfs diff=lfs merge=lfs -text
42
+ Tess-2.0-Yi-34B-200K-Q3_K_M.gguf filter=lfs diff=lfs merge=lfs -text
43
+ Tess-2.0-Yi-34B-200K-Q3_K_S.gguf filter=lfs diff=lfs merge=lfs -text
44
+ Tess-2.0-Yi-34B-200K-Q4_0.gguf filter=lfs diff=lfs merge=lfs -text
45
+ Tess-2.0-Yi-34B-200K-Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text
46
+ Tess-2.0-Yi-34B-200K-Q4_K_S.gguf filter=lfs diff=lfs merge=lfs -text
47
+ Tess-2.0-Yi-34B-200K-Q5_0.gguf filter=lfs diff=lfs merge=lfs -text
48
+ Tess-2.0-Yi-34B-200K-Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text
49
+ Tess-2.0-Yi-34B-200K-Q5_K_S.gguf filter=lfs diff=lfs merge=lfs -text
50
+ Tess-2.0-Yi-34B-200K-Q6_K.gguf filter=lfs diff=lfs merge=lfs -text
51
+ Tess-2.0-Yi-34B-200K-Q8_0.gguf filter=lfs diff=lfs merge=lfs -text
52
+ Tess-2.0-Yi-34B-200K-fp16.gguf/Tess-2.0-Yi-34B-200K-fp16_part_a filter=lfs diff=lfs merge=lfs -text
53
+ Tess-2.0-Yi-34B-200K-fp16.gguf/Tess-2.0-Yi-34B-200K-fp16_part_b filter=lfs diff=lfs merge=lfs -text
README.md ADDED
@@ -0,0 +1,36 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: other
3
+ license_name: yi-34b
4
+ license_link: https://huggingface.co/01-ai/Yi-34B-200K/blob/main/LICENSE
5
+ quantized_by: bartowski
6
+ pipeline_tag: text-generation
7
+ ---
8
+
9
+ ## Llamacpp Quantizations of Tess-2.0-Yi-34B-200K
10
+
11
+ Using <a href="https://github.com/ggerganov/llama.cpp/">llama.cpp</a> release <a href="https://github.com/ggerganov/llama.cpp/releases/tag/b2536">b2536</a> for quantization.
12
+
13
+ Original model: https://huggingface.co/migtissera/Tess-2.0-Yi-34B-200K
14
+
15
+ Download a file (not the whole branch) from below:
16
+
17
+ | Filename | Quant type | File Size | Description |
18
+ | -------- | ---------- | --------- | ----------- |
19
+ | [Tess-2.0-Yi-34B-200K-Q8_0.gguf](https://huggingface.co/bartowski/Tess-2.0-Yi-34B-200K-GGUF/blob/main/Tess-2.0-Yi-34B-200K-Q8_0.gguf) | Q8_0 | 36.54GB | Extremely high quality, generally unneeded but max available quant. |
20
+ | [Tess-2.0-Yi-34B-200K-Q6_K.gguf](https://huggingface.co/bartowski/Tess-2.0-Yi-34B-200K-GGUF/blob/main/Tess-2.0-Yi-34B-200K-Q6_K.gguf) | Q6_K | 28.21GB | Very high quality, near perfect, *recommended*. |
21
+ | [Tess-2.0-Yi-34B-200K-Q5_K_M.gguf](https://huggingface.co/bartowski/Tess-2.0-Yi-34B-200K-GGUF/blob/main/Tess-2.0-Yi-34B-200K-Q5_K_M.gguf) | Q5_K_M | 24.32GB | High quality, very usable. |
22
+ | [Tess-2.0-Yi-34B-200K-Q5_K_S.gguf](https://huggingface.co/bartowski/Tess-2.0-Yi-34B-200K-GGUF/blob/main/Tess-2.0-Yi-34B-200K-Q5_K_S.gguf) | Q5_K_S | 23.70GB | High quality, very usable. |
23
+ | [Tess-2.0-Yi-34B-200K-Q5_0.gguf](https://huggingface.co/bartowski/Tess-2.0-Yi-34B-200K-GGUF/blob/main/Tess-2.0-Yi-34B-200K-Q5_0.gguf) | Q5_0 | 23.70GB | High quality, older format, generally not recommended. |
24
+ | [Tess-2.0-Yi-34B-200K-Q4_K_M.gguf](https://huggingface.co/bartowski/Tess-2.0-Yi-34B-200K-GGUF/blob/main/Tess-2.0-Yi-34B-200K-Q4_K_M.gguf) | Q4_K_M | 20.65GB | Good quality, uses about 4.83 bits per weight. |
25
+ | [Tess-2.0-Yi-34B-200K-Q4_K_S.gguf](https://huggingface.co/bartowski/Tess-2.0-Yi-34B-200K-GGUF/blob/main/Tess-2.0-Yi-34B-200K-Q4_K_S.gguf) | Q4_K_S | 19.59GB | Slightly lower quality with small space savings. |
26
+ | [Tess-2.0-Yi-34B-200K-IQ4_NL.gguf](https://huggingface.co/bartowski/Tess-2.0-Yi-34B-200K-GGUF/blob/main/Tess-2.0-Yi-34B-200K-IQ4_NL.gguf) | IQ4_NL | 19.65GB | Decent quality, similar to Q4_K_S, new method of quanting, |
27
+ | [Tess-2.0-Yi-34B-200K-IQ4_XS.gguf](https://huggingface.co/bartowski/Tess-2.0-Yi-34B-200K-GGUF/blob/main/Tess-2.0-Yi-34B-200K-IQ4_XS.gguf) | IQ4_XS | 18.63GB | Decent quality, new method with similar performance to Q4. |
28
+ | [Tess-2.0-Yi-34B-200K-Q4_0.gguf](https://huggingface.co/bartowski/Tess-2.0-Yi-34B-200K-GGUF/blob/main/Tess-2.0-Yi-34B-200K-Q4_0.gguf) | Q4_0 | 19.46GB | Decent quality, older format, generally not recommended. |
29
+ | [Tess-2.0-Yi-34B-200K-Q3_K_L.gguf](https://huggingface.co/bartowski/Tess-2.0-Yi-34B-200K-GGUF/blob/main/Tess-2.0-Yi-34B-200K-Q3_K_L.gguf) | Q3_K_L | 18.13GB | Lower quality but usable, good for low RAM availability. |
30
+ | [Tess-2.0-Yi-34B-200K-Q3_K_M.gguf](https://huggingface.co/bartowski/Tess-2.0-Yi-34B-200K-GGUF/blob/main/Tess-2.0-Yi-34B-200K-Q3_K_M.gguf) | Q3_K_M | 16.65GB | Even lower quality. |
31
+ | [Tess-2.0-Yi-34B-200K-IQ3_M.gguf](https://huggingface.co/bartowski/Tess-2.0-Yi-34B-200K-GGUF/blob/main/Tess-2.0-Yi-34B-200K-IQ3_M.gguf) | IQ3_M | 15.56GB | Medium-low quality, new method with decent performance. |
32
+ | [Tess-2.0-Yi-34B-200K-IQ3_S.gguf](https://huggingface.co/bartowski/Tess-2.0-Yi-34B-200K-GGUF/blob/main/Tess-2.0-Yi-34B-200K-IQ3_S.gguf) | IQ3_S | 15.01GB | Lower quality, new method with decent performance, recommended over Q3 quants. |
33
+ | [Tess-2.0-Yi-34B-200K-Q3_K_S.gguf](https://huggingface.co/bartowski/Tess-2.0-Yi-34B-200K-GGUF/blob/main/Tess-2.0-Yi-34B-200K-Q3_K_S.gguf) | Q3_K_S | 14.96GB | Low quality, not recommended. |
34
+ | [Tess-2.0-Yi-34B-200K-Q2_K.gguf](https://huggingface.co/bartowski/Tess-2.0-Yi-34B-200K-GGUF/blob/main/Tess-2.0-Yi-34B-200K-Q2_K.gguf) | Q2_K | 12.82GB | Extremely low quality, *not* recommended.
35
+
36
+ Want to support my work? Visit my ko-fi page here: https://ko-fi.com/bartowski
Tess-2.0-Yi-34B-200K-IQ3_M.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:b764bcffdf5adf959f8cf3aa4c8cf0c3d8fb13d89b3800b4b06c6c3a4bbd69da
3
+ size 15564717728
Tess-2.0-Yi-34B-200K-IQ3_S.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:9bc8300d72016002a01f8945c1bb4e68f0a8daf3d23c7689de39a368cafed9fb
3
+ size 15018802848
Tess-2.0-Yi-34B-200K-IQ4_NL.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:427e3bb94327bc4fe4d95dcd7ec9e98c3a5ecd0ffac3213e11c232fe074159ae
3
+ size 19650049536
Tess-2.0-Yi-34B-200K-IQ4_XS.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:f36544c63af9f0b60ad06f19902880a85d7d9bd2f35db3128d6dd170f5f31d64
3
+ size 18635633728
Tess-2.0-Yi-34B-200K-Q2_K.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:7dd36b5f17db6a65652a1f0ff694b08dd388eaeb86039975d318ede80bfd8899
3
+ size 12825250016
Tess-2.0-Yi-34B-200K-Q3_K_L.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:b7cd3203596c462fa053035f0ea4bfbd4fb78f21d74100f647a1252333bcfc7a
3
+ size 18139463328
Tess-2.0-Yi-34B-200K-Q3_K_M.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:202ee4f07f8be6ede668bee24f63ea2521b6f86fdebe199cc0fb076943b050f3
3
+ size 16654941856
Tess-2.0-Yi-34B-200K-Q3_K_S.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:36e25a7152f2c54357f33c2c2bac5d5707440b506a29910c73c50ace90011561
3
+ size 14960311968
Tess-2.0-Yi-34B-200K-Q4_0.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:7796c84b2d3a7dcc3fa1b305c6c36b4638544def3da7fa60a799ca7c3db0ffbc
3
+ size 19466548736
Tess-2.0-Yi-34B-200K-Q4_K_M.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:848de22d2f0a5f975a30776c301c25f967d48f985316755aa297aa9edcdc9c3d
3
+ size 20658730496
Tess-2.0-Yi-34B-200K-Q4_K_S.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:35bc234304615c94e4526b3b2a27aef4c3d147fc8499700740f23d4b24616a0f
3
+ size 19598669312
Tess-2.0-Yi-34B-200K-Q5_0.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:a7ba2dbb86f265bbb50c9208a113fd5e0d4bdd398af26137bd1918a3d8cb4dbe
3
+ size 23707712768
Tess-2.0-Yi-34B-200K-Q5_K_M.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:b37a12e60ae600cc63825ae73b25b29ebcba72fa7fe1b52f41bbfa3a1a088c19
3
+ size 24321867008
Tess-2.0-Yi-34B-200K-Q5_K_S.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:a7db44af05970569bca9b0724125f50ad306961cdd950053490660e643ccd570
3
+ size 23707712768
Tess-2.0-Yi-34B-200K-Q6_K.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:fdbeda7dc70e25bc5bcec695c634662b3e9e650bea129f3df2a8e4d343e441ea
3
+ size 28213949568
Tess-2.0-Yi-34B-200K-Q8_0.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:f0b9e45895a5144c6da4f3f32b0b249d91f26c842a70f68d9c0596586a6bcfb2
3
+ size 36542312320
Tess-2.0-Yi-34B-200K-fp16.gguf/Tess-2.0-Yi-34B-200K-fp16_part_a ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:c77299d1ba0a04c61e9d3cbdec764ad77b5112b2f2947a0efc16cd4156040748
3
+ size 34390567969
Tess-2.0-Yi-34B-200K-fp16.gguf/Tess-2.0-Yi-34B-200K-fp16_part_b ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:7cbfaaab317cea6478c768eb5d65b35778c28c9ee717bfae2f204e353b13f128
3
+ size 34390567967
Tess-2.0-Yi-34B-200K-fp16.gguf/combine.sh ADDED
@@ -0,0 +1,2 @@
 
 
 
1
+ #!/bin/bash
2
+ cat Tess-2.0-Yi-34B-200K-fp16_part_* > "Tess-2.0-Yi-34B-200K-fp16.gguf"