legraphista
commited on
Commit
β’
b70336a
1
Parent(s):
191dedd
Upload README.md with huggingface_hub
Browse files
README.md
CHANGED
@@ -20,6 +20,7 @@ tags:
|
|
20 |
- 4bit
|
21 |
- 3bit
|
22 |
- 2bit
|
|
|
23 |
---
|
24 |
|
25 |
# Yuan2-M32-hf-IMat-GGUF
|
@@ -27,7 +28,7 @@ _Llama.cpp imatrix quantization of IEITYuan/Yuan2-M32-hf_
|
|
27 |
|
28 |
Original Model: [IEITYuan/Yuan2-M32-hf](https://huggingface.co/IEITYuan/Yuan2-M32-hf)
|
29 |
Original dtype: `BF16` (`bfloat16`)
|
30 |
-
Quantized by: [https://github.com/chong000/3rd_party/tree/main](https://github.com/chong000/3rd_party/tree/main)
|
31 |
IMatrix dataset: [here](https://gist.githubusercontent.com/legraphista/d6d93f1a254bcfc58e0af3777eaec41e/raw/d380e7002cea4a51c33fffd47db851942754e7cc/imatrix.calibration.medium.raw)
|
32 |
|
33 |
- [Files](#files)
|
@@ -73,8 +74,12 @@ Link: [here](https://huggingface.co/legraphista/Yuan2-M32-hf-IMat-GGUF/blob/main
|
|
73 |
| [Yuan2-M32-hf.Q3_K.gguf](https://huggingface.co/legraphista/Yuan2-M32-hf-IMat-GGUF/blob/main/Yuan2-M32-hf.Q3_K.gguf) | Q3_K | 19.54GB | β
Available | π’ IMatrix | π¦ No
|
74 |
| [Yuan2-M32-hf.Q3_K_L.gguf](https://huggingface.co/legraphista/Yuan2-M32-hf-IMat-GGUF/blob/main/Yuan2-M32-hf.Q3_K_L.gguf) | Q3_K_L | 21.14GB | β
Available | π’ IMatrix | π¦ No
|
75 |
| [Yuan2-M32-hf.Q3_K_S.gguf](https://huggingface.co/legraphista/Yuan2-M32-hf-IMat-GGUF/blob/main/Yuan2-M32-hf.Q3_K_S.gguf) | Q3_K_S | 17.71GB | β
Available | π’ IMatrix | π¦ No
|
|
|
76 |
| [Yuan2-M32-hf.Q2_K.gguf](https://huggingface.co/legraphista/Yuan2-M32-hf-IMat-GGUF/blob/main/Yuan2-M32-hf.Q2_K.gguf) | Q2_K | 15.02GB | β
Available | π’ IMatrix | π¦ No
|
77 |
| [Yuan2-M32-hf.Q2_K_S.gguf](https://huggingface.co/legraphista/Yuan2-M32-hf-IMat-GGUF/blob/main/Yuan2-M32-hf.Q2_K_S.gguf) | Q2_K_S | 14.05GB | β
Available | π’ IMatrix | π¦ No
|
|
|
|
|
|
|
78 |
|
79 |
|
80 |
## Downloading using huggingface-cli
|
|
|
20 |
- 4bit
|
21 |
- 3bit
|
22 |
- 2bit
|
23 |
+
- 1bit
|
24 |
---
|
25 |
|
26 |
# Yuan2-M32-hf-IMat-GGUF
|
|
|
28 |
|
29 |
Original Model: [IEITYuan/Yuan2-M32-hf](https://huggingface.co/IEITYuan/Yuan2-M32-hf)
|
30 |
Original dtype: `BF16` (`bfloat16`)
|
31 |
+
Quantized by: [https://github.com/chong000/3rd_party/tree/main](https://github.com/chong000/3rd_party/tree/main)
|
32 |
IMatrix dataset: [here](https://gist.githubusercontent.com/legraphista/d6d93f1a254bcfc58e0af3777eaec41e/raw/d380e7002cea4a51c33fffd47db851942754e7cc/imatrix.calibration.medium.raw)
|
33 |
|
34 |
- [Files](#files)
|
|
|
74 |
| [Yuan2-M32-hf.Q3_K.gguf](https://huggingface.co/legraphista/Yuan2-M32-hf-IMat-GGUF/blob/main/Yuan2-M32-hf.Q3_K.gguf) | Q3_K | 19.54GB | β
Available | π’ IMatrix | π¦ No
|
75 |
| [Yuan2-M32-hf.Q3_K_L.gguf](https://huggingface.co/legraphista/Yuan2-M32-hf-IMat-GGUF/blob/main/Yuan2-M32-hf.Q3_K_L.gguf) | Q3_K_L | 21.14GB | β
Available | π’ IMatrix | π¦ No
|
76 |
| [Yuan2-M32-hf.Q3_K_S.gguf](https://huggingface.co/legraphista/Yuan2-M32-hf-IMat-GGUF/blob/main/Yuan2-M32-hf.Q3_K_S.gguf) | Q3_K_S | 17.71GB | β
Available | π’ IMatrix | π¦ No
|
77 |
+
| [Yuan2-M32-hf.IQ3_XXS.gguf](https://huggingface.co/legraphista/Yuan2-M32-hf-IMat-GGUF/blob/main/Yuan2-M32-hf.IQ3_XXS.gguf) | IQ3_XXS | 15.91GB | β
Available | π’ IMatrix | π¦ No
|
78 |
| [Yuan2-M32-hf.Q2_K.gguf](https://huggingface.co/legraphista/Yuan2-M32-hf-IMat-GGUF/blob/main/Yuan2-M32-hf.Q2_K.gguf) | Q2_K | 15.02GB | β
Available | π’ IMatrix | π¦ No
|
79 |
| [Yuan2-M32-hf.Q2_K_S.gguf](https://huggingface.co/legraphista/Yuan2-M32-hf-IMat-GGUF/blob/main/Yuan2-M32-hf.Q2_K_S.gguf) | Q2_K_S | 14.05GB | β
Available | π’ IMatrix | π¦ No
|
80 |
+
| Yuan2-M32-hf.IQ2_XS | IQ2_XS | - | β³ Processing | π’ IMatrix | -
|
81 |
+
| Yuan2-M32-hf.IQ2_XXS | IQ2_XXS | - | β³ Processing | π’ IMatrix | -
|
82 |
+
| Yuan2-M32-hf.IQ1_S | IQ1_S | - | β³ Processing | π’ IMatrix | -
|
83 |
|
84 |
|
85 |
## Downloading using huggingface-cli
|