Hugging Face's logo

bartowski
/
c4ai-command-r-v01-GGUF

Text Generation
Transformers
GGUF
Inference Endpoints
Model card Files Community
2
bartowski commited on
Commit
7ff1d6d
•
1 Parent(s): 701cb9c

Llamacpp quants

Browse files
Files changed (17) hide show
  1. .gitattributes +14 -0
  2. README.md +42 -0
  3. c4ai-command-r-v01-Q2_K.gguf +3 -0
  4. c4ai-command-r-v01-Q3_K_L.gguf +3 -0
  5. c4ai-command-r-v01-Q3_K_M.gguf +3 -0
  6. c4ai-command-r-v01-Q3_K_S.gguf +3 -0
  7. c4ai-command-r-v01-Q4_0.gguf +3 -0
  8. c4ai-command-r-v01-Q4_K_M.gguf +3 -0
  9. c4ai-command-r-v01-Q4_K_S.gguf +3 -0
  10. c4ai-command-r-v01-Q5_0.gguf +3 -0
  11. c4ai-command-r-v01-Q5_K_M.gguf +3 -0
  12. c4ai-command-r-v01-Q5_K_S.gguf +3 -0
  13. c4ai-command-r-v01-Q6_K.gguf +3 -0
  14. c4ai-command-r-v01-Q8_0.gguf +3 -0
  15. c4ai-command-r-v01-fp16.gguf/c4ai-command-r-v01-fp16_part_a +3 -0
  16. c4ai-command-r-v01-fp16.gguf/c4ai-command-r-v01-fp16_part_b +3 -0
  17. c4ai-command-r-v01-fp16.gguf/combine.sh +2 -0
.gitattributes CHANGED
@@ -33,3 +33,17 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
36
+ c4ai-command-r-v01-Q2_K.gguf filter=lfs diff=lfs merge=lfs -text
37
+ c4ai-command-r-v01-Q3_K_L.gguf filter=lfs diff=lfs merge=lfs -text
38
+ c4ai-command-r-v01-Q3_K_M.gguf filter=lfs diff=lfs merge=lfs -text
39
+ c4ai-command-r-v01-Q3_K_S.gguf filter=lfs diff=lfs merge=lfs -text
40
+ c4ai-command-r-v01-Q4_0.gguf filter=lfs diff=lfs merge=lfs -text
41
+ c4ai-command-r-v01-Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text
42
+ c4ai-command-r-v01-Q4_K_S.gguf filter=lfs diff=lfs merge=lfs -text
43
+ c4ai-command-r-v01-Q5_0.gguf filter=lfs diff=lfs merge=lfs -text
44
+ c4ai-command-r-v01-Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text
45
+ c4ai-command-r-v01-Q5_K_S.gguf filter=lfs diff=lfs merge=lfs -text
46
+ c4ai-command-r-v01-Q6_K.gguf filter=lfs diff=lfs merge=lfs -text
47
+ c4ai-command-r-v01-Q8_0.gguf filter=lfs diff=lfs merge=lfs -text
48
+ c4ai-command-r-v01-fp16.gguf/c4ai-command-r-v01-fp16_part_a filter=lfs diff=lfs merge=lfs -text
49
+ c4ai-command-r-v01-fp16.gguf/c4ai-command-r-v01-fp16_part_b filter=lfs diff=lfs merge=lfs -text
README.md ADDED
@@ -0,0 +1,42 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ library_name: transformers
3
+ language:
4
+ - en
5
+ - fr
6
+ - de
7
+ - es
8
+ - it
9
+ - pt
10
+ - ja
11
+ - ko
12
+ - zh
13
+ - ar
14
+ license: cc-by-nc-4.0
15
+ quantized_by: bartowski
16
+ pipeline_tag: text-generation
17
+ ---
18
+
19
+ ## Llamacpp Quantizations of c4ai-command-r-v01
20
+
21
+ Using <a href="https://github.com/ggerganov/llama.cpp/">llama.cpp</a> release <a href="https://github.com/ggerganov/llama.cpp/releases/tag/b2440">b2440</a> for quantization.
22
+
23
+ Original model: https://huggingface.co/CohereForAI/c4ai-command-r-v01
24
+
25
+ Download a file (not the whole branch) from below:
26
+
27
+ | Filename | Quant type | File Size | Description |
28
+ | -------- | ---------- | --------- | ----------- |
29
+ | [c4ai-command-r-v01-Q8_0.gguf](https://huggingface.co/bartowski/c4ai-command-r-v01-GGUF/blob/main/c4ai-command-r-v01-Q8_0.gguf) | Q8_0 | 37.17GB | Extremely high quality, generally unneeded but max available quant. |
30
+ | [c4ai-command-r-v01-Q6_K.gguf](https://huggingface.co/bartowski/c4ai-command-r-v01-GGUF/blob/main/c4ai-command-r-v01-Q6_K.gguf) | Q6_K | 28.70GB | Very high quality, near perfect, *recommended*. |
31
+ | [c4ai-command-r-v01-Q5_K_M.gguf](https://huggingface.co/bartowski/c4ai-command-r-v01-GGUF/blob/main/c4ai-command-r-v01-Q5_K_M.gguf) | Q5_K_M | 25.00GB | High quality, very usable. |
32
+ | [c4ai-command-r-v01-Q5_K_S.gguf](https://huggingface.co/bartowski/c4ai-command-r-v01-GGUF/blob/main/c4ai-command-r-v01-Q5_K_S.gguf) | Q5_K_S | 24.33GB | High quality, very usable. |
33
+ | [c4ai-command-r-v01-Q5_0.gguf](https://huggingface.co/bartowski/c4ai-command-r-v01-GGUF/blob/main/c4ai-command-r-v01-Q5_0.gguf) | Q5_0 | 24.33GB | High quality, older format, generally not recommended. |
34
+ | [c4ai-command-r-v01-Q4_K_M.gguf](https://huggingface.co/bartowski/c4ai-command-r-v01-GGUF/blob/main/c4ai-command-r-v01-Q4_K_M.gguf) | Q4_K_M | 21.52GB | Good quality, similar to 4.25 bpw. |
35
+ | [c4ai-command-r-v01-Q4_K_S.gguf](https://huggingface.co/bartowski/c4ai-command-r-v01-GGUF/blob/main/c4ai-command-r-v01-Q4_K_S.gguf) | Q4_K_S | 20.37GB | Slightly lower quality with small space savings. |
36
+ | [c4ai-command-r-v01-Q4_0.gguf](https://huggingface.co/bartowski/c4ai-command-r-v01-GGUF/blob/main/c4ai-command-r-v01-Q4_0.gguf) | Q4_0 | 20.22GB | Decent quality, older format, generally not recommended. |
37
+ | [c4ai-command-r-v01-Q3_K_L.gguf](https://huggingface.co/bartowski/c4ai-command-r-v01-GGUF/blob/main/c4ai-command-r-v01-Q3_K_L.gguf) | Q3_K_L | 19.14GB | Lower quality but usable, good for low RAM availability. |
38
+ | [c4ai-command-r-v01-Q3_K_M.gguf](https://huggingface.co/bartowski/c4ai-command-r-v01-GGUF/blob/main/c4ai-command-r-v01-Q3_K_M.gguf) | Q3_K_M | 17.61GB | Even lower quality. |
39
+ | [c4ai-command-r-v01-Q3_K_S.gguf](https://huggingface.co/bartowski/c4ai-command-r-v01-GGUF/blob/main/c4ai-command-r-v01-Q3_K_S.gguf) | Q3_K_S | 15.86GB | Low quality, not recommended. |
40
+ | [c4ai-command-r-v01-Q2_K.gguf](https://huggingface.co/bartowski/c4ai-command-r-v01-GGUF/blob/main/c4ai-command-r-v01-Q2_K.gguf) | Q2_K | 13.81GB | Extremely low quality, *not* recommended.
41
+
42
+ Want to support my work? Visit my ko-fi page here: https://ko-fi.com/bartowski
c4ai-command-r-v01-Q2_K.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:7a1cc81d5f8cd99b11ed92ce1a70dfca257e81f8ace7b391f5a033b4c227a779
3
+ size 13817386816
c4ai-command-r-v01-Q3_K_L.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:2ae3ddadba72c022f3f07df99bb2759baba052240ab3397f4044d433616b1933
3
+ size 19149395776
c4ai-command-r-v01-Q3_K_M.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:fafdefa03c759df7c70f570f46fb43f3ea7d303c99fed5152a543fddb5b6c3e4
3
+ size 17618474816
c4ai-command-r-v01-Q3_K_S.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:3d039ff1205bec9294d70d4e3cfedac50c0cad5738e5429a1772a40573c21917
3
+ size 15862110016
c4ai-command-r-v01-Q4_0.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:7a44b7495c25b7b70c26761898ca018beb350b5994dad47abc1480fbcd794ed0
3
+ size 20229429056
c4ai-command-r-v01-Q4_K_M.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:66f5676ac64dc2eadac689a1b09bfda2edd5dbc77500d9596edac933857a7aec
3
+ size 21527041856
c4ai-command-r-v01-Q4_K_S.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:7642d93b29750e5c90b86af774cc65eb22555e6973c016f30391145de3d53b0f
3
+ size 20378326848
c4ai-command-r-v01-Q5_0.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:434ae56bc4fe992d0f2208a7c4ee0638bab85f504e6dfd01094023601ba77875
3
+ size 24339846976
c4ai-command-r-v01-Q5_K_M.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:1a59aeb034b64e430d25bc9f2b29d9f2cc658af38670fae36226585603da8ecc
3
+ size 25008314176
c4ai-command-r-v01-Q5_K_S.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:be0f4f70054db710b309d9da83198c4cb8676ff90795be2c9f796d40af394853
3
+ size 24339846976
c4ai-command-r-v01-Q6_K.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:7cb7b3500598862fe32db5fe5c927792a5144512da9e236895103a3b160e3456
3
+ size 28707166016
c4ai-command-r-v01-Q8_0.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:bc49062c243a188ddb957e8d777800501e7ea95254334e196836c4284bddc299
3
+ size 37179004736
c4ai-command-r-v01-fp16.gguf/c4ai-command-r-v01-fp16_part_a ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:41a78dc45b9784aa0a1c500605455c6abbe70ad490f60c2a963d135f25bceaa4
3
+ size 34986609553
c4ai-command-r-v01-fp16.gguf/c4ai-command-r-v01-fp16_part_b ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:d60e80d726b6e7f81a6ffd41e7bc98651ef87dd96e477fdc4f487834a9f9a922
3
+ size 34986609551
c4ai-command-r-v01-fp16.gguf/combine.sh ADDED
@@ -0,0 +1,2 @@
 
 
 
1
+ #!/bin/bash
2
+ cat c4ai-command-r-v01-fp16_part_* > "c4ai-command-r-v01-fp16.gguf"