Hugging Face's logo

bartowski
/
Excalibur-7b-GGUF

Text Generation
Transformers
GGUF
mergekit
Merge
Inference Endpoints
Model card Files Community
bartowski commited on
Commit
e488f42
•
1 Parent(s): 91b0f3f

Llamacpp quants

Browse files
Files changed (18) hide show
  1. .gitattributes +16 -0
  2. Excalibur-7b-IQ3_M.gguf +3 -0
  3. Excalibur-7b-IQ3_S.gguf +3 -0
  4. Excalibur-7b-IQ4_NL.gguf +3 -0
  5. Excalibur-7b-IQ4_XS.gguf +3 -0
  6. Excalibur-7b-Q2_K.gguf +3 -0
  7. Excalibur-7b-Q3_K_L.gguf +3 -0
  8. Excalibur-7b-Q3_K_M.gguf +3 -0
  9. Excalibur-7b-Q3_K_S.gguf +3 -0
  10. Excalibur-7b-Q4_0.gguf +3 -0
  11. Excalibur-7b-Q4_K_M.gguf +3 -0
  12. Excalibur-7b-Q4_K_S.gguf +3 -0
  13. Excalibur-7b-Q5_0.gguf +3 -0
  14. Excalibur-7b-Q5_K_M.gguf +3 -0
  15. Excalibur-7b-Q5_K_S.gguf +3 -0
  16. Excalibur-7b-Q6_K.gguf +3 -0
  17. Excalibur-7b-Q8_0.gguf +3 -0
  18. README.md +41 -0
.gitattributes CHANGED
@@ -33,3 +33,19 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
36
+ Excalibur-7b-IQ3_M.gguf filter=lfs diff=lfs merge=lfs -text
37
+ Excalibur-7b-IQ3_S.gguf filter=lfs diff=lfs merge=lfs -text
38
+ Excalibur-7b-IQ4_NL.gguf filter=lfs diff=lfs merge=lfs -text
39
+ Excalibur-7b-IQ4_XS.gguf filter=lfs diff=lfs merge=lfs -text
40
+ Excalibur-7b-Q2_K.gguf filter=lfs diff=lfs merge=lfs -text
41
+ Excalibur-7b-Q3_K_L.gguf filter=lfs diff=lfs merge=lfs -text
42
+ Excalibur-7b-Q3_K_M.gguf filter=lfs diff=lfs merge=lfs -text
43
+ Excalibur-7b-Q3_K_S.gguf filter=lfs diff=lfs merge=lfs -text
44
+ Excalibur-7b-Q4_0.gguf filter=lfs diff=lfs merge=lfs -text
45
+ Excalibur-7b-Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text
46
+ Excalibur-7b-Q4_K_S.gguf filter=lfs diff=lfs merge=lfs -text
47
+ Excalibur-7b-Q5_0.gguf filter=lfs diff=lfs merge=lfs -text
48
+ Excalibur-7b-Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text
49
+ Excalibur-7b-Q5_K_S.gguf filter=lfs diff=lfs merge=lfs -text
50
+ Excalibur-7b-Q6_K.gguf filter=lfs diff=lfs merge=lfs -text
51
+ Excalibur-7b-Q8_0.gguf filter=lfs diff=lfs merge=lfs -text
Excalibur-7b-IQ3_M.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:a5a6f16ba1a82bd0fc2e62eff887324c79a3556a799fe6587d443d575327be8f
3
+ size 3284891424
Excalibur-7b-IQ3_S.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:07529d793b797652ab3046d7bb8f110929622d2363dd014e2eb4b6112c0a4f1e
3
+ size 3182393120
Excalibur-7b-IQ4_NL.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:f7ebe5279ab5357a1700f890290c103948ff075e3635a6ba840f1c72b7c38e8c
3
+ size 4155053856
Excalibur-7b-IQ4_XS.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:34034ef5f89e619390078637f166a1832e88c1dc21ea4862dea8cb8dced3dcdb
3
+ size 3944388384
Excalibur-7b-Q2_K.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:5713dd881e6ceb2dc57063dfc6892a146e7bd616678363e8bc618097fde2a547
3
+ size 2719242016
Excalibur-7b-Q3_K_L.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:a09eeaf1899f25671ca6ae6f26086eb002109b235d5c3a8892582208abc1e9c9
3
+ size 3822024480
Excalibur-7b-Q3_K_M.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:8805609f15b4946b7e2b5a818e33423e54cde46148b29e9f72d395863a38ed6a
3
+ size 3518986016
Excalibur-7b-Q3_K_S.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:a166486807b2c77c5770dccd9a5f9bbba9b486790c9173c7874b2939c8e87e8c
3
+ size 3164567328
Excalibur-7b-Q4_0.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:93944ff580bf71851edbf3fc203850be58dff9e6d28ef3b2b0002039c64f25c7
3
+ size 4108916512
Excalibur-7b-Q4_K_M.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:c122f6e6ea8b35983c3d521667a42feabc5b73997dcf2458253a2df7bda2f9ec
3
+ size 4368439072
Excalibur-7b-Q4_K_S.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:77621787abde99fd38ab4e5229921c79a587a82fef1f2b6b168092d204257d91
3
+ size 4140373792
Excalibur-7b-Q5_0.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:d0c27348add0f13a61986645595e354694b5bf1e16530528de108694090d8908
3
+ size 4997715744
Excalibur-7b-Q5_K_M.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:251e2339a5aeca8aa2979d8250b2e624408802e03669ea67943c8fec75e78adb
3
+ size 5131409184
Excalibur-7b-Q5_K_S.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:2e0a265c385071c55e931d69009e7b78d6b8724b2ea1280709e99316854ff048
3
+ size 4997715744
Excalibur-7b-Q6_K.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:9de9639a73331c40492c9d60130c8cd8ed65f5d11d42f485647f0dba6f386477
3
+ size 5942064928
Excalibur-7b-Q8_0.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:1d349110d0f920729bd34ae48bc9bdef4fcb6678f263277f513063e1365fb3c2
3
+ size 7695857440
README.md ADDED
@@ -0,0 +1,41 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ base_model:
3
+ - ibm/merlinite-7b
4
+ - InferenceIllusionist/Magic-Dolphin-7b
5
+ - SanjiWatsuki/Kunoichi-DPO-v2-7B
6
+ - mlabonne/Monarch-7B
7
+ - bardsai/jaskier-7b-dpo-v6.1
8
+
9
+ library_name: transformers
10
+ tags:
11
+ - mergekit
12
+ - merge
13
+ license: apache-2.0
14
+ quantized_by: bartowski
15
+ pipeline_tag: text-generation
16
+ ---
17
+
18
+ ## Llamacpp Quantizations of Excalibur-7b
19
+
20
+ Using <a href="https://github.com/ggerganov/llama.cpp/">llama.cpp</a> release <a href="https://github.com/ggerganov/llama.cpp/releases/tag/b2440">b2440</a> for quantization.
21
+
22
+ Original model: https://huggingface.co/InferenceIllusionist/Excalibur-7b
23
+
24
+ Download a file (not the whole branch) from below:
25
+
26
+ | Filename | Quant type | File Size | Description |
27
+ | -------- | ---------- | --------- | ----------- |
28
+ | [Excalibur-7b-Q8_0.gguf](https://huggingface.co/bartowski/Excalibur-7b-GGUF/blob/main/Excalibur-7b-Q8_0.gguf) | Q8_0 | 7.69GB | Extremely high quality, generally unneeded but max available quant. |
29
+ | [Excalibur-7b-Q6_K.gguf](https://huggingface.co/bartowski/Excalibur-7b-GGUF/blob/main/Excalibur-7b-Q6_K.gguf) | Q6_K | 5.94GB | Very high quality, near perfect, *recommended*. |
30
+ | [Excalibur-7b-Q5_K_M.gguf](https://huggingface.co/bartowski/Excalibur-7b-GGUF/blob/main/Excalibur-7b-Q5_K_M.gguf) | Q5_K_M | 5.13GB | High quality, very usable. |
31
+ | [Excalibur-7b-Q5_K_S.gguf](https://huggingface.co/bartowski/Excalibur-7b-GGUF/blob/main/Excalibur-7b-Q5_K_S.gguf) | Q5_K_S | 4.99GB | High quality, very usable. |
32
+ | [Excalibur-7b-Q5_0.gguf](https://huggingface.co/bartowski/Excalibur-7b-GGUF/blob/main/Excalibur-7b-Q5_0.gguf) | Q5_0 | 4.99GB | High quality, older format, generally not recommended. |
33
+ | [Excalibur-7b-Q4_K_M.gguf](https://huggingface.co/bartowski/Excalibur-7b-GGUF/blob/main/Excalibur-7b-Q4_K_M.gguf) | Q4_K_M | 4.36GB | Good quality, similar to 4.25 bpw. |
34
+ | [Excalibur-7b-Q4_K_S.gguf](https://huggingface.co/bartowski/Excalibur-7b-GGUF/blob/main/Excalibur-7b-Q4_K_S.gguf) | Q4_K_S | 4.14GB | Slightly lower quality with small space savings. |
35
+ | [Excalibur-7b-Q4_0.gguf](https://huggingface.co/bartowski/Excalibur-7b-GGUF/blob/main/Excalibur-7b-Q4_0.gguf) | Q4_0 | 4.10GB | Decent quality, older format, generally not recommended. |
36
+ | [Excalibur-7b-Q3_K_L.gguf](https://huggingface.co/bartowski/Excalibur-7b-GGUF/blob/main/Excalibur-7b-Q3_K_L.gguf) | Q3_K_L | 3.82GB | Lower quality but usable, good for low RAM availability. |
37
+ | [Excalibur-7b-Q3_K_M.gguf](https://huggingface.co/bartowski/Excalibur-7b-GGUF/blob/main/Excalibur-7b-Q3_K_M.gguf) | Q3_K_M | 3.51GB | Even lower quality. |
38
+ | [Excalibur-7b-Q3_K_S.gguf](https://huggingface.co/bartowski/Excalibur-7b-GGUF/blob/main/Excalibur-7b-Q3_K_S.gguf) | Q3_K_S | 3.16GB | Low quality, not recommended. |
39
+ | [Excalibur-7b-Q2_K.gguf](https://huggingface.co/bartowski/Excalibur-7b-GGUF/blob/main/Excalibur-7b-Q2_K.gguf) | Q2_K | 2.71GB | Extremely low quality, *not* recommended.
40
+
41
+ Want to support my work? Visit my ko-fi page here: https://ko-fi.com/bartowski