Hugging Face's logo

bartowski
/
DIBT-Mistral-7B-QLoRA-GGUF

Text Generation
GGUF
Inference Endpoints
conversational
Model card Files Community
bartowski commited on
Commit
d272924
•
1 Parent(s): 1ea2ffb

Llamacpp quants

Browse files
Files changed (15) hide show
  1. .gitattributes +13 -0
  2. DIBT-Mistral-7B-QLoRA-Q2_K.gguf +3 -0
  3. DIBT-Mistral-7B-QLoRA-Q3_K_L.gguf +3 -0
  4. DIBT-Mistral-7B-QLoRA-Q3_K_M.gguf +3 -0
  5. DIBT-Mistral-7B-QLoRA-Q3_K_S.gguf +3 -0
  6. DIBT-Mistral-7B-QLoRA-Q4_0.gguf +3 -0
  7. DIBT-Mistral-7B-QLoRA-Q4_K_M.gguf +3 -0
  8. DIBT-Mistral-7B-QLoRA-Q4_K_S.gguf +3 -0
  9. DIBT-Mistral-7B-QLoRA-Q5_0.gguf +3 -0
  10. DIBT-Mistral-7B-QLoRA-Q5_K_M.gguf +3 -0
  11. DIBT-Mistral-7B-QLoRA-Q5_K_S.gguf +3 -0
  12. DIBT-Mistral-7B-QLoRA-Q6_K.gguf +3 -0
  13. DIBT-Mistral-7B-QLoRA-Q8_0.gguf +3 -0
  14. DIBT-Mistral-7B-QLoRA-fp16.gguf +3 -0
  15. README.md +31 -0
.gitattributes CHANGED
@@ -33,3 +33,16 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
 
 
 
 
 
 
 
 
 
 
 
 
 
 
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
36
+ DIBT-Mistral-7B-QLoRA-Q2_K.gguf filter=lfs diff=lfs merge=lfs -text
37
+ DIBT-Mistral-7B-QLoRA-Q3_K_L.gguf filter=lfs diff=lfs merge=lfs -text
38
+ DIBT-Mistral-7B-QLoRA-Q3_K_M.gguf filter=lfs diff=lfs merge=lfs -text
39
+ DIBT-Mistral-7B-QLoRA-Q3_K_S.gguf filter=lfs diff=lfs merge=lfs -text
40
+ DIBT-Mistral-7B-QLoRA-Q4_0.gguf filter=lfs diff=lfs merge=lfs -text
41
+ DIBT-Mistral-7B-QLoRA-Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text
42
+ DIBT-Mistral-7B-QLoRA-Q4_K_S.gguf filter=lfs diff=lfs merge=lfs -text
43
+ DIBT-Mistral-7B-QLoRA-Q5_0.gguf filter=lfs diff=lfs merge=lfs -text
44
+ DIBT-Mistral-7B-QLoRA-Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text
45
+ DIBT-Mistral-7B-QLoRA-Q5_K_S.gguf filter=lfs diff=lfs merge=lfs -text
46
+ DIBT-Mistral-7B-QLoRA-Q6_K.gguf filter=lfs diff=lfs merge=lfs -text
47
+ DIBT-Mistral-7B-QLoRA-Q8_0.gguf filter=lfs diff=lfs merge=lfs -text
48
+ DIBT-Mistral-7B-QLoRA-fp16.gguf filter=lfs diff=lfs merge=lfs -text
DIBT-Mistral-7B-QLoRA-Q2_K.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:dfef0379480e4dcc009415125dea63b0578c0eed00774901a6eea2343762a324
3
+ size 2719242240
DIBT-Mistral-7B-QLoRA-Q3_K_L.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:a5c84320273f23c584fde9a48bc38043eb1e9180b97b2bbc04223e414a28e4c4
3
+ size 3822024704
DIBT-Mistral-7B-QLoRA-Q3_K_M.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:094b6b0561363f53ce849ad6bacf089e6dca42e9c19d7eeaa62c784e803be833
3
+ size 3518986240
DIBT-Mistral-7B-QLoRA-Q3_K_S.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:79661d0b6d0b8cda4600ae7bfc368943592bdf76907d3bebcda65ac5cca72250
3
+ size 3164567552
DIBT-Mistral-7B-QLoRA-Q4_0.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:186500314e7556d12bb69a4d1a973c12004c59fa28231e2c8d97e107f3e92c18
3
+ size 4108916736
DIBT-Mistral-7B-QLoRA-Q4_K_M.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:d37bdf426be6422ce337a26dc0e60116b08e582da1d1d361c56496067052e8ba
3
+ size 4368439296
DIBT-Mistral-7B-QLoRA-Q4_K_S.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:f2f7205531313fed73ca4c1ca56e911d65f4e4a6b35ce101425a31479f6bbed2
3
+ size 4140374016
DIBT-Mistral-7B-QLoRA-Q5_0.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:c735c3e50af91feeba547f58d7ca0bc46dbcd15d285e9382fc0900cafc61ad45
3
+ size 4997715968
DIBT-Mistral-7B-QLoRA-Q5_K_M.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:7fb156cf2c814a94c4fb7915cc63d7476f424b9d54572bd65883f2a9e451e35c
3
+ size 5131409408
DIBT-Mistral-7B-QLoRA-Q5_K_S.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:2544a8a5d812cfaf552ce68e04659000a633910fe8594dbb4c9539cda8474208
3
+ size 4997715968
DIBT-Mistral-7B-QLoRA-Q6_K.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:b0577e3366d1a7da85117cbeff4fcb40bcb3a8af10715216bbfee4a67301b6f3
3
+ size 5942065152
DIBT-Mistral-7B-QLoRA-Q8_0.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:8d260459cf1d205e1ff85450d7d1443c1c145bc3dcb0c5af23c093acb20d5ab2
3
+ size 7695857664
DIBT-Mistral-7B-QLoRA-fp16.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:353c775c689a19e7d07897c9e3acec567871806014a8f7f9c6ede554dc274ee2
3
+ size 14484731872
README.md ADDED
@@ -0,0 +1,31 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ # For reference on model card metadata, see the spec: https://github.com/huggingface/hub-docs/blob/main/modelcard.md?plain=1
3
+ # Doc / guide: https://huggingface.co/docs/hub/model-cards
4
+ quantized_by: bartowski
5
+ pipeline_tag: text-generation
6
+ ---
7
+
8
+ ## Llamacpp Quantizations of DIBT-Mistral-7B-QLoRA
9
+
10
+ Using <a href="https://github.com/ggerganov/llama.cpp/">llama.cpp</a> release <a href="https://github.com/ggerganov/llama.cpp/releases/tag/b2440">b2440</a> for quantization.
11
+
12
+ Original model: https://huggingface.co/Locutusque/DIBT-Mistral-7B-QLoRA
13
+
14
+ Download a file (not the whole branch) from below:
15
+
16
+ | Filename | Quant type | File Size | Description |
17
+ | -------- | ---------- | --------- | ----------- |
18
+ | [DIBT-Mistral-7B-QLoRA-Q8_0.gguf](https://huggingface.co/bartowski/DIBT-Mistral-7B-QLoRA-GGUF/blob/main/DIBT-Mistral-7B-QLoRA-Q8_0.gguf) | Q8_0 | 7.69GB | Extremely high quality, generally unneeded but max available quant. |
19
+ | [DIBT-Mistral-7B-QLoRA-Q6_K.gguf](https://huggingface.co/bartowski/DIBT-Mistral-7B-QLoRA-GGUF/blob/main/DIBT-Mistral-7B-QLoRA-Q6_K.gguf) | Q6_K | 5.94GB | Very high quality, near perfect, *recommended*. |
20
+ | [DIBT-Mistral-7B-QLoRA-Q5_K_M.gguf](https://huggingface.co/bartowski/DIBT-Mistral-7B-QLoRA-GGUF/blob/main/DIBT-Mistral-7B-QLoRA-Q5_K_M.gguf) | Q5_K_M | 5.13GB | High quality, very usable. |
21
+ | [DIBT-Mistral-7B-QLoRA-Q5_K_S.gguf](https://huggingface.co/bartowski/DIBT-Mistral-7B-QLoRA-GGUF/blob/main/DIBT-Mistral-7B-QLoRA-Q5_K_S.gguf) | Q5_K_S | 4.99GB | High quality, very usable. |
22
+ | [DIBT-Mistral-7B-QLoRA-Q5_0.gguf](https://huggingface.co/bartowski/DIBT-Mistral-7B-QLoRA-GGUF/blob/main/DIBT-Mistral-7B-QLoRA-Q5_0.gguf) | Q5_0 | 4.99GB | High quality, older format, generally not recommended. |
23
+ | [DIBT-Mistral-7B-QLoRA-Q4_K_M.gguf](https://huggingface.co/bartowski/DIBT-Mistral-7B-QLoRA-GGUF/blob/main/DIBT-Mistral-7B-QLoRA-Q4_K_M.gguf) | Q4_K_M | 4.36GB | Good quality, similar to 4.25 bpw. |
24
+ | [DIBT-Mistral-7B-QLoRA-Q4_K_S.gguf](https://huggingface.co/bartowski/DIBT-Mistral-7B-QLoRA-GGUF/blob/main/DIBT-Mistral-7B-QLoRA-Q4_K_S.gguf) | Q4_K_S | 4.14GB | Slightly lower quality with small space savings. |
25
+ | [DIBT-Mistral-7B-QLoRA-Q4_0.gguf](https://huggingface.co/bartowski/DIBT-Mistral-7B-QLoRA-GGUF/blob/main/DIBT-Mistral-7B-QLoRA-Q4_0.gguf) | Q4_0 | 4.10GB | Decent quality, older format, generally not recommended. |
26
+ | [DIBT-Mistral-7B-QLoRA-Q3_K_L.gguf](https://huggingface.co/bartowski/DIBT-Mistral-7B-QLoRA-GGUF/blob/main/DIBT-Mistral-7B-QLoRA-Q3_K_L.gguf) | Q3_K_L | 3.82GB | Lower quality but usable, good for low RAM availability. |
27
+ | [DIBT-Mistral-7B-QLoRA-Q3_K_M.gguf](https://huggingface.co/bartowski/DIBT-Mistral-7B-QLoRA-GGUF/blob/main/DIBT-Mistral-7B-QLoRA-Q3_K_M.gguf) | Q3_K_M | 3.51GB | Even lower quality. |
28
+ | [DIBT-Mistral-7B-QLoRA-Q3_K_S.gguf](https://huggingface.co/bartowski/DIBT-Mistral-7B-QLoRA-GGUF/blob/main/DIBT-Mistral-7B-QLoRA-Q3_K_S.gguf) | Q3_K_S | 3.16GB | Low quality, not recommended. |
29
+ | [DIBT-Mistral-7B-QLoRA-Q2_K.gguf](https://huggingface.co/bartowski/DIBT-Mistral-7B-QLoRA-GGUF/blob/main/DIBT-Mistral-7B-QLoRA-Q2_K.gguf) | Q2_K | 2.71GB | Extremely low quality, *not* recommended.
30
+
31
+ Want to support my work? Visit my ko-fi page here: https://ko-fi.com/bartowski