Hugging Face's logo

bartowski
/
openbuddy-mistral-7b-v19.1-4k-GGUF

Text Generation
GGUF
Inference Endpoints
Model card Files Community
1
bartowski commited on
Commit
0b04a69
•
1 Parent(s): f4302f6

Llamacpp quants

Browse files
Files changed (15) hide show
  1. .gitattributes +13 -0
  2. README.md +30 -0
  3. openbuddy-mistral-7b-v19.1-4k-Q2_K.gguf +3 -0
  4. openbuddy-mistral-7b-v19.1-4k-Q3_K_L.gguf +3 -0
  5. openbuddy-mistral-7b-v19.1-4k-Q3_K_M.gguf +3 -0
  6. openbuddy-mistral-7b-v19.1-4k-Q3_K_S.gguf +3 -0
  7. openbuddy-mistral-7b-v19.1-4k-Q4_0.gguf +3 -0
  8. openbuddy-mistral-7b-v19.1-4k-Q4_K_M.gguf +3 -0
  9. openbuddy-mistral-7b-v19.1-4k-Q4_K_S.gguf +3 -0
  10. openbuddy-mistral-7b-v19.1-4k-Q5_0.gguf +3 -0
  11. openbuddy-mistral-7b-v19.1-4k-Q5_K_M.gguf +3 -0
  12. openbuddy-mistral-7b-v19.1-4k-Q5_K_S.gguf +3 -0
  13. openbuddy-mistral-7b-v19.1-4k-Q6_K.gguf +3 -0
  14. openbuddy-mistral-7b-v19.1-4k-Q8_0.gguf +3 -0
  15. openbuddy-mistral-7b-v19.1-4k-fp16.gguf +3 -0
.gitattributes CHANGED
@@ -33,3 +33,16 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
 
 
 
 
 
 
 
 
 
 
 
 
 
 
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
36
+ openbuddy-mistral-7b-v19.1-4k-Q2_K.gguf filter=lfs diff=lfs merge=lfs -text
37
+ openbuddy-mistral-7b-v19.1-4k-Q3_K_L.gguf filter=lfs diff=lfs merge=lfs -text
38
+ openbuddy-mistral-7b-v19.1-4k-Q3_K_M.gguf filter=lfs diff=lfs merge=lfs -text
39
+ openbuddy-mistral-7b-v19.1-4k-Q3_K_S.gguf filter=lfs diff=lfs merge=lfs -text
40
+ openbuddy-mistral-7b-v19.1-4k-Q4_0.gguf filter=lfs diff=lfs merge=lfs -text
41
+ openbuddy-mistral-7b-v19.1-4k-Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text
42
+ openbuddy-mistral-7b-v19.1-4k-Q4_K_S.gguf filter=lfs diff=lfs merge=lfs -text
43
+ openbuddy-mistral-7b-v19.1-4k-Q5_0.gguf filter=lfs diff=lfs merge=lfs -text
44
+ openbuddy-mistral-7b-v19.1-4k-Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text
45
+ openbuddy-mistral-7b-v19.1-4k-Q5_K_S.gguf filter=lfs diff=lfs merge=lfs -text
46
+ openbuddy-mistral-7b-v19.1-4k-Q6_K.gguf filter=lfs diff=lfs merge=lfs -text
47
+ openbuddy-mistral-7b-v19.1-4k-Q8_0.gguf filter=lfs diff=lfs merge=lfs -text
48
+ openbuddy-mistral-7b-v19.1-4k-fp16.gguf filter=lfs diff=lfs merge=lfs -text
README.md ADDED
@@ -0,0 +1,30 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: apache-2.0
3
+ quantized_by: bartowski
4
+ pipeline_tag: text-generation
5
+ ---
6
+
7
+ ## Llamacpp Quantizations of openbuddy-mistral-7b-v19.1-4k
8
+
9
+ Using <a href="https://github.com/ggerganov/llama.cpp/commit/fa974646e1a2024fc7dc9e6f27cf1f2f5d4a3763">llama.cpp commit fa97464</a> for quantization.
10
+
11
+ Original model: https://huggingface.co/OpenBuddy/openbuddy-mistral-7b-v19.1-4k
12
+
13
+ Download a file (not the whole branch) from below:
14
+
15
+ | Filename | Quant type | File Size | Description |
16
+ | -------- | ---------- | --------- | ----------- |
17
+ | [openbuddy-mistral-7b-v19.1-4k-Q8_0.gguf](https://huggingface.co/bartowski/openbuddy-mistral-7b-v19.1-4k-GGUF/blob/main/openbuddy-mistral-7b-v19.1-4k-Q8_0.gguf) | Q8_0 | 7.73GB | Extremely high quality, generally unneeded but max available quant. |
18
+ | [openbuddy-mistral-7b-v19.1-4k-Q6_K.gguf](https://huggingface.co/bartowski/openbuddy-mistral-7b-v19.1-4k-GGUF/blob/main/openbuddy-mistral-7b-v19.1-4k-Q6_K.gguf) | Q6_K | 5.97GB | Very high quality, near perfect, *recommended*. |
19
+ | [openbuddy-mistral-7b-v19.1-4k-Q5_K_M.gguf](https://huggingface.co/bartowski/openbuddy-mistral-7b-v19.1-4k-GGUF/blob/main/openbuddy-mistral-7b-v19.1-4k-Q5_K_M.gguf) | Q5_K_M | 5.15GB | High quality, very usable. |
20
+ | [openbuddy-mistral-7b-v19.1-4k-Q5_K_S.gguf](https://huggingface.co/bartowski/openbuddy-mistral-7b-v19.1-4k-GGUF/blob/main/openbuddy-mistral-7b-v19.1-4k-Q5_K_S.gguf) | Q5_K_S | 5.02GB | High quality, very usable. |
21
+ | [openbuddy-mistral-7b-v19.1-4k-Q5_0.gguf](https://huggingface.co/bartowski/openbuddy-mistral-7b-v19.1-4k-GGUF/blob/main/openbuddy-mistral-7b-v19.1-4k-Q5_0.gguf) | Q5_0 | 5.02GB | High quality, older format, generally not recommended. |
22
+ | [openbuddy-mistral-7b-v19.1-4k-Q4_K_M.gguf](https://huggingface.co/bartowski/openbuddy-mistral-7b-v19.1-4k-GGUF/blob/main/openbuddy-mistral-7b-v19.1-4k-Q4_K_M.gguf) | Q4_K_M | 4.39GB | Good quality, similar to 4.25 bpw. |
23
+ | [openbuddy-mistral-7b-v19.1-4k-Q4_K_S.gguf](https://huggingface.co/bartowski/openbuddy-mistral-7b-v19.1-4k-GGUF/blob/main/openbuddy-mistral-7b-v19.1-4k-Q4_K_S.gguf) | Q4_K_S | 4.16GB | Slightly lower quality with small space savings. |
24
+ | [openbuddy-mistral-7b-v19.1-4k-Q4_0.gguf](https://huggingface.co/bartowski/openbuddy-mistral-7b-v19.1-4k-GGUF/blob/main/openbuddy-mistral-7b-v19.1-4k-Q4_0.gguf) | Q4_0 | 4.13GB | Decent quality, older format, generally not recommended. |
25
+ | [openbuddy-mistral-7b-v19.1-4k-Q3_K_L.gguf](https://huggingface.co/bartowski/openbuddy-mistral-7b-v19.1-4k-GGUF/blob/main/openbuddy-mistral-7b-v19.1-4k-Q3_K_L.gguf) | Q3_K_L | 3.84GB | Lower quality but usable, good for low RAM availability. |
26
+ | [openbuddy-mistral-7b-v19.1-4k-Q3_K_M.gguf](https://huggingface.co/bartowski/openbuddy-mistral-7b-v19.1-4k-GGUF/blob/main/openbuddy-mistral-7b-v19.1-4k-Q3_K_M.gguf) | Q3_K_M | 3.54GB | Even lower quality. |
27
+ | [openbuddy-mistral-7b-v19.1-4k-Q3_K_S.gguf](https://huggingface.co/bartowski/openbuddy-mistral-7b-v19.1-4k-GGUF/blob/main/openbuddy-mistral-7b-v19.1-4k-Q3_K_S.gguf) | Q3_K_S | 3.18GB | Low quality, not recommended. |
28
+ | [openbuddy-mistral-7b-v19.1-4k-Q2_K.gguf](https://huggingface.co/bartowski/openbuddy-mistral-7b-v19.1-4k-GGUF/blob/main/openbuddy-mistral-7b-v19.1-4k-Q2_K.gguf) | Q2_K | 2.74GB | Extremely low quality, *not* recommended.
29
+
30
+ Want to support my work? Visit my ko-fi page here: https://ko-fi.com/bartowski
openbuddy-mistral-7b-v19.1-4k-Q2_K.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:f3c635cedaa10a438a0a5f3ff79b2f338a6e4ce2c8bc149eedfb644dc0a486ab
3
+ size 2741006112
openbuddy-mistral-7b-v19.1-4k-Q3_K_L.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:6eec889f97ae453714c45aaead83638cfdc5eac44a0717f147c4d9c53bbfd22c
3
+ size 3845705504
openbuddy-mistral-7b-v19.1-4k-Q3_K_M.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:32c75d294e718803a19cb7a591151fd2ec17ec69b4e33f4292e1063e4c8d5c29
3
+ size 3542667040
openbuddy-mistral-7b-v19.1-4k-Q3_K_S.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:1936a1bfb87e7ad85dfdb08857a0fd15ac596ab8d4358f98d54b80e219e4fcd5
3
+ size 3188248352
openbuddy-mistral-7b-v19.1-4k-Q4_0.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:9ab65951ded367d69c43f34f1356d5471ca177a59e46191a98b805b848a69f47
3
+ size 4135104288
openbuddy-mistral-7b-v19.1-4k-Q4_K_M.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:2aa53bdbd5d1e858fb788d6f744e499e957a1c7afeca78b66eb02b166bb6a59d
3
+ size 4394626848
openbuddy-mistral-7b-v19.1-4k-Q4_K_S.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:2c286971023903f7ba052410b156a5510c368539b1c71d371d3a3d63358dd4b8
3
+ size 4166561568
openbuddy-mistral-7b-v19.1-4k-Q5_0.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:52c2fea80f52b7a7071d314c584c72116963d23b8201a8601cbac3bd971b925e
3
+ size 5026262816
openbuddy-mistral-7b-v19.1-4k-Q5_K_M.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:882267e6ef70faa030d033824b5ddda00772afb537cd8f6711890bcb062f4df3
3
+ size 5159956256
openbuddy-mistral-7b-v19.1-4k-Q5_K_S.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:fc8d07583d0b7affd5ff9398aae7ac94ca9e5c8d543521c5e79be885936ac1a6
3
+ size 5026262816
openbuddy-mistral-7b-v19.1-4k-Q6_K.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:bc111b52041530eb4ba8bc84765c8041b485eb99d998028911573150205d628b
3
+ size 5973118752
openbuddy-mistral-7b-v19.1-4k-Q8_0.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:19c61c8256244a2b96daaa114aabcaf8f4b52b0388b6696e96cf0e3d04ed43e1
3
+ size 7736053536
openbuddy-mistral-7b-v19.1-4k-fp16.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:a5f1a135837ff90ad0cf77e541b54047fa1003582473d90747d056c5d3ca7271
3
+ size 14560317184