Upload README.md with huggingface_hub
Browse files
README.md
CHANGED
@@ -27,7 +27,7 @@ tags:
|
|
27 |
_Llama.cpp imatrix quantization of LLM360/K2-Chat_
|
28 |
|
29 |
Original Model: [LLM360/K2-Chat](https://huggingface.co/LLM360/K2-Chat)
|
30 |
-
Original dtype: `
|
31 |
Quantized by: llama.cpp [b3051](https://github.com/ggerganov/llama.cpp/releases/tag/b3051)
|
32 |
IMatrix dataset: [here](https://gist.githubusercontent.com/bartowski1182/eb213dccb3571f863da82e99418f81e8/raw/b2869d80f5c16fd7082594248e80144677736635/calibration_datav3.txt)
|
33 |
|
@@ -53,7 +53,7 @@ Link: [here](https://huggingface.co/legraphista/K2-Chat-IMat-GGUF/blob/main/imat
|
|
53 |
### Common Quants
|
54 |
| Filename | Quant type | File Size | Status | Uses IMatrix | Is Split |
|
55 |
| -------- | ---------- | --------- | ------ | ------------ | -------- |
|
56 |
-
| K2-Chat.Q8_0 | Q8_0 |
|
57 |
| K2-Chat.Q6_K | Q6_K | - | β³ Processing | βͺ Static | -
|
58 |
| K2-Chat.Q4_K | Q4_K | - | β³ Processing | π’ IMatrix | -
|
59 |
| K2-Chat.Q3_K | Q3_K | - | β³ Processing | π’ IMatrix | -
|
@@ -63,9 +63,8 @@ Link: [here](https://huggingface.co/legraphista/K2-Chat-IMat-GGUF/blob/main/imat
|
|
63 |
### All Quants
|
64 |
| Filename | Quant type | File Size | Status | Uses IMatrix | Is Split |
|
65 |
| -------- | ---------- | --------- | ------ | ------------ | -------- |
|
66 |
-
| K2-Chat.BF16 | BF16 | - | β³ Processing | βͺ Static | -
|
67 |
| K2-Chat.FP16 | F16 | - | β³ Processing | βͺ Static | -
|
68 |
-
| K2-Chat.Q8_0 | Q8_0 |
|
69 |
| K2-Chat.Q6_K | Q6_K | - | β³ Processing | βͺ Static | -
|
70 |
| K2-Chat.Q5_K | Q5_K | - | β³ Processing | βͺ Static | -
|
71 |
| K2-Chat.Q5_K_S | Q5_K_S | - | β³ Processing | βͺ Static | -
|
|
|
27 |
_Llama.cpp imatrix quantization of LLM360/K2-Chat_
|
28 |
|
29 |
Original Model: [LLM360/K2-Chat](https://huggingface.co/LLM360/K2-Chat)
|
30 |
+
Original dtype: `FP16` (`float16`)
|
31 |
Quantized by: llama.cpp [b3051](https://github.com/ggerganov/llama.cpp/releases/tag/b3051)
|
32 |
IMatrix dataset: [here](https://gist.githubusercontent.com/bartowski1182/eb213dccb3571f863da82e99418f81e8/raw/b2869d80f5c16fd7082594248e80144677736635/calibration_datav3.txt)
|
33 |
|
|
|
53 |
### Common Quants
|
54 |
| Filename | Quant type | File Size | Status | Uses IMatrix | Is Split |
|
55 |
| -------- | ---------- | --------- | ------ | ------------ | -------- |
|
56 |
+
| [K2-Chat.Q8_0/*](https://huggingface.co/legraphista/K2-Chat-IMat-GGUF/tree/main/K2-Chat.Q8_0) | Q8_0 | 69.37GB | β
Available | βͺ Static | β Yes
|
57 |
| K2-Chat.Q6_K | Q6_K | - | β³ Processing | βͺ Static | -
|
58 |
| K2-Chat.Q4_K | Q4_K | - | β³ Processing | π’ IMatrix | -
|
59 |
| K2-Chat.Q3_K | Q3_K | - | β³ Processing | π’ IMatrix | -
|
|
|
63 |
### All Quants
|
64 |
| Filename | Quant type | File Size | Status | Uses IMatrix | Is Split |
|
65 |
| -------- | ---------- | --------- | ------ | ------------ | -------- |
|
|
|
66 |
| K2-Chat.FP16 | F16 | - | β³ Processing | βͺ Static | -
|
67 |
+
| [K2-Chat.Q8_0/*](https://huggingface.co/legraphista/K2-Chat-IMat-GGUF/tree/main/K2-Chat.Q8_0) | Q8_0 | 69.37GB | β
Available | βͺ Static | β Yes
|
68 |
| K2-Chat.Q6_K | Q6_K | - | β³ Processing | βͺ Static | -
|
69 |
| K2-Chat.Q5_K | Q5_K | - | β³ Processing | βͺ Static | -
|
70 |
| K2-Chat.Q5_K_S | Q5_K_S | - | β³ Processing | βͺ Static | -
|