Hugging Face's logo

bartowski
/
Tess-10.7B-v2.0-GGUF

Text Generation
Transformers
GGUF
mergekit
Merge
Inference Endpoints
Model card Files Community
bartowski commited on
Commit
4686e82
•
1 Parent(s): 273a1f3

Llamacpp quants

Browse files
Files changed (18) hide show
  1. .gitattributes +16 -0
  2. README.md +39 -0
  3. Tess-10.7B-v2.0-IQ3_M.gguf +3 -0
  4. Tess-10.7B-v2.0-IQ3_S.gguf +3 -0
  5. Tess-10.7B-v2.0-IQ4_NL.gguf +3 -0
  6. Tess-10.7B-v2.0-IQ4_XS.gguf +3 -0
  7. Tess-10.7B-v2.0-Q2_K.gguf +3 -0
  8. Tess-10.7B-v2.0-Q3_K_L.gguf +3 -0
  9. Tess-10.7B-v2.0-Q3_K_M.gguf +3 -0
  10. Tess-10.7B-v2.0-Q3_K_S.gguf +3 -0
  11. Tess-10.7B-v2.0-Q4_0.gguf +3 -0
  12. Tess-10.7B-v2.0-Q4_K_M.gguf +3 -0
  13. Tess-10.7B-v2.0-Q4_K_S.gguf +3 -0
  14. Tess-10.7B-v2.0-Q5_0.gguf +3 -0
  15. Tess-10.7B-v2.0-Q5_K_M.gguf +3 -0
  16. Tess-10.7B-v2.0-Q5_K_S.gguf +3 -0
  17. Tess-10.7B-v2.0-Q6_K.gguf +3 -0
  18. Tess-10.7B-v2.0-Q8_0.gguf +3 -0
.gitattributes CHANGED
@@ -33,3 +33,19 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
36
+ Tess-10.7B-v2.0-IQ3_M.gguf filter=lfs diff=lfs merge=lfs -text
37
+ Tess-10.7B-v2.0-IQ3_S.gguf filter=lfs diff=lfs merge=lfs -text
38
+ Tess-10.7B-v2.0-IQ4_NL.gguf filter=lfs diff=lfs merge=lfs -text
39
+ Tess-10.7B-v2.0-IQ4_XS.gguf filter=lfs diff=lfs merge=lfs -text
40
+ Tess-10.7B-v2.0-Q2_K.gguf filter=lfs diff=lfs merge=lfs -text
41
+ Tess-10.7B-v2.0-Q3_K_L.gguf filter=lfs diff=lfs merge=lfs -text
42
+ Tess-10.7B-v2.0-Q3_K_M.gguf filter=lfs diff=lfs merge=lfs -text
43
+ Tess-10.7B-v2.0-Q3_K_S.gguf filter=lfs diff=lfs merge=lfs -text
44
+ Tess-10.7B-v2.0-Q4_0.gguf filter=lfs diff=lfs merge=lfs -text
45
+ Tess-10.7B-v2.0-Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text
46
+ Tess-10.7B-v2.0-Q4_K_S.gguf filter=lfs diff=lfs merge=lfs -text
47
+ Tess-10.7B-v2.0-Q5_0.gguf filter=lfs diff=lfs merge=lfs -text
48
+ Tess-10.7B-v2.0-Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text
49
+ Tess-10.7B-v2.0-Q5_K_S.gguf filter=lfs diff=lfs merge=lfs -text
50
+ Tess-10.7B-v2.0-Q6_K.gguf filter=lfs diff=lfs merge=lfs -text
51
+ Tess-10.7B-v2.0-Q8_0.gguf filter=lfs diff=lfs merge=lfs -text
README.md ADDED
@@ -0,0 +1,39 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: apache-2.0
3
+ base_model: []
4
+ library_name: transformers
5
+ tags:
6
+ - mergekit
7
+ - merge
8
+ pipeline_tag: text-generation
9
+ quantized_by: bartowski
10
+ ---
11
+
12
+ ## Llamacpp Quantizations of Tess-10.7B-v2.0
13
+
14
+ Using <a href="https://github.com/ggerganov/llama.cpp/">llama.cpp</a> release <a href="https://github.com/ggerganov/llama.cpp/releases/tag/b2536">b2536</a> for quantization.
15
+
16
+ Original model: https://huggingface.co/Joseph717171/Tess-10.7B-v2.0
17
+
18
+ Download a file (not the whole branch) from below:
19
+
20
+ | Filename | Quant type | File Size | Description |
21
+ | -------- | ---------- | --------- | ----------- |
22
+ | [Tess-10.7B-v2.0-Q8_0.gguf](https://huggingface.co/bartowski/Tess-10.7B-v2.0-GGUF/blob/main/Tess-10.7B-v2.0-Q8_0.gguf) | Q8_0 | 11.40GB | Extremely high quality, generally unneeded but max available quant. |
23
+ | [Tess-10.7B-v2.0-Q6_K.gguf](https://huggingface.co/bartowski/Tess-10.7B-v2.0-GGUF/blob/main/Tess-10.7B-v2.0-Q6_K.gguf) | Q6_K | 8.80GB | Very high quality, near perfect, *recommended*. |
24
+ | [Tess-10.7B-v2.0-Q5_K_M.gguf](https://huggingface.co/bartowski/Tess-10.7B-v2.0-GGUF/blob/main/Tess-10.7B-v2.0-Q5_K_M.gguf) | Q5_K_M | 7.59GB | High quality, very usable. |
25
+ | [Tess-10.7B-v2.0-Q5_K_S.gguf](https://huggingface.co/bartowski/Tess-10.7B-v2.0-GGUF/blob/main/Tess-10.7B-v2.0-Q5_K_S.gguf) | Q5_K_S | 7.39GB | High quality, very usable. |
26
+ | [Tess-10.7B-v2.0-Q5_0.gguf](https://huggingface.co/bartowski/Tess-10.7B-v2.0-GGUF/blob/main/Tess-10.7B-v2.0-Q5_0.gguf) | Q5_0 | 7.39GB | High quality, older format, generally not recommended. |
27
+ | [Tess-10.7B-v2.0-Q4_K_M.gguf](https://huggingface.co/bartowski/Tess-10.7B-v2.0-GGUF/blob/main/Tess-10.7B-v2.0-Q4_K_M.gguf) | Q4_K_M | 6.46GB | Good quality, uses about 4.83 bits per weight. |
28
+ | [Tess-10.7B-v2.0-Q4_K_S.gguf](https://huggingface.co/bartowski/Tess-10.7B-v2.0-GGUF/blob/main/Tess-10.7B-v2.0-Q4_K_S.gguf) | Q4_K_S | 6.11GB | Slightly lower quality with small space savings. |
29
+ | [Tess-10.7B-v2.0-IQ4_NL.gguf](https://huggingface.co/bartowski/Tess-10.7B-v2.0-GGUF/blob/main/Tess-10.7B-v2.0-IQ4_NL.gguf) | IQ4_NL | 6.14GB | Decent quality, similar to Q4_K_S, new method of quanting, |
30
+ | [Tess-10.7B-v2.0-IQ4_XS.gguf](https://huggingface.co/bartowski/Tess-10.7B-v2.0-GGUF/blob/main/Tess-10.7B-v2.0-IQ4_XS.gguf) | IQ4_XS | 5.82GB | Decent quality, new method with similar performance to Q4. |
31
+ | [Tess-10.7B-v2.0-Q4_0.gguf](https://huggingface.co/bartowski/Tess-10.7B-v2.0-GGUF/blob/main/Tess-10.7B-v2.0-Q4_0.gguf) | Q4_0 | 6.07GB | Decent quality, older format, generally not recommended. |
32
+ | [Tess-10.7B-v2.0-Q3_K_L.gguf](https://huggingface.co/bartowski/Tess-10.7B-v2.0-GGUF/blob/main/Tess-10.7B-v2.0-Q3_K_L.gguf) | Q3_K_L | 5.65GB | Lower quality but usable, good for low RAM availability. |
33
+ | [Tess-10.7B-v2.0-Q3_K_M.gguf](https://huggingface.co/bartowski/Tess-10.7B-v2.0-GGUF/blob/main/Tess-10.7B-v2.0-Q3_K_M.gguf) | Q3_K_M | 5.19GB | Even lower quality. |
34
+ | [Tess-10.7B-v2.0-IQ3_M.gguf](https://huggingface.co/bartowski/Tess-10.7B-v2.0-GGUF/blob/main/Tess-10.7B-v2.0-IQ3_M.gguf) | IQ3_M | 4.84GB | Medium-low quality, new method with decent performance. |
35
+ | [Tess-10.7B-v2.0-IQ3_S.gguf](https://huggingface.co/bartowski/Tess-10.7B-v2.0-GGUF/blob/main/Tess-10.7B-v2.0-IQ3_S.gguf) | IQ3_S | 4.69GB | Lower quality, new method with decent performance, recommended over Q3 quants. |
36
+ | [Tess-10.7B-v2.0-Q3_K_S.gguf](https://huggingface.co/bartowski/Tess-10.7B-v2.0-GGUF/blob/main/Tess-10.7B-v2.0-Q3_K_S.gguf) | Q3_K_S | 4.66GB | Low quality, not recommended. |
37
+ | [Tess-10.7B-v2.0-Q2_K.gguf](https://huggingface.co/bartowski/Tess-10.7B-v2.0-GGUF/blob/main/Tess-10.7B-v2.0-Q2_K.gguf) | Q2_K | 4.00GB | Extremely low quality, *not* recommended.
38
+
39
+ Want to support my work? Visit my ko-fi page here: https://ko-fi.com/bartowski
Tess-10.7B-v2.0-IQ3_M.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:57ce46a687602898a33f627a93912ab41bc24d796f8e304a130838e43f93f019
3
+ size 4845050560
Tess-10.7B-v2.0-IQ3_S.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:974eb65210b4afba388422227f94c4a34ee4844df7221a0447881eca86c6df98
3
+ size 4691303104
Tess-10.7B-v2.0-IQ4_NL.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:8d8182909374479f530c811eb652b17a7dce2d795ce27303228a98d1d4c35939
3
+ size 6141590208
Tess-10.7B-v2.0-IQ4_XS.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:62b724acfb2392cd3d6c42a03540639c4249dc3ae4a72f38fcefb2f9b34f1ae3
3
+ size 5827640000
Tess-10.7B-v2.0-Q2_K.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:045ec86506cd62bbf5d25387637dfbd5f486986949a0724647af60bf2bd45178
3
+ size 4003232448
Tess-10.7B-v2.0-Q3_K_L.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:ce10f58303284490c94cb4ef9adf068235b448f9f4c795e7e92c6904b952be9b
3
+ size 5650750144
Tess-10.7B-v2.0-Q3_K_M.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:63e91d00fd1b258dfde26271ac53875723140fc853fc36f259bb22da8603eab6
3
+ size 5195668160
Tess-10.7B-v2.0-Q3_K_S.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:11b768d4fdb311258324cad8c2a1bc9ca7a100100cc0971b540501e553ec8b2e
3
+ size 4664564416
Tess-10.7B-v2.0-Q4_0.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:c1dff3c4c95b43f35b7d7103c806c75cf2d940f0f810016e8e9dd9df279b23a1
3
+ size 6072384192
Tess-10.7B-v2.0-Q4_K_M.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:b74443676f2fc4c93d0fb29df8d1243fcc1ef760612f1464aa0c90adf4782300
3
+ size 6461668032
Tess-10.7B-v2.0-Q4_K_S.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:20b7580a7ef03eaf7923f63955e3349fdbf4978032672a08680cd2cb1b222d58
3
+ size 6118521536
Tess-10.7B-v2.0-Q5_0.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:cdd58076284be160e4edc68431b645943323490709e42e69338026e7e8325ecd
3
+ size 7397391040
Tess-10.7B-v2.0-Q5_K_M.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:b27cdaa72c1430c32df445d9d413c2c1d971314cc7d030a905f2f25ced0e1240
3
+ size 7597931200
Tess-10.7B-v2.0-Q5_K_S.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:04532f0391aba3bee9fa7f179a864484f534c52badf62dd70ebc9f2fe4dd07a4
3
+ size 7397391040
Tess-10.7B-v2.0-Q6_K.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:3c86a449db7ae7fc3fa0c5e56a14db4fbb68406a481e3178aed9451a8a4faaed
3
+ size 8805210816
Tess-10.7B-v2.0-Q8_0.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:d672a0117d58bbcefe57edf25e7db489dc2d0b090181ee656658fdd5bb03dcf9
3
+ size 11404155584