Hugging Face's logo

secretmoon
/
WoonaV1.2-9b-GGUF-Imatrix

Text Generation
Transformers
GGUF
Russian
unsloth
sft
pony
MyLittlePony
Russian
Lora
Inference Endpoints
imatrix
conversational
Model card Files Community
secretmoon commited on
Commit
670b394
•
1 Parent(s): 40d9cfd

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +9 -9
README.md CHANGED
@@ -32,14 +32,14 @@ GGUF imatrix quants of **[AlexBefest/WoonaV1.2-9b](https://huggingface.co/AlexBe
32
 
33
  | Name | Quant method | Bits | Size | Min RAM required | Use case |
34
  | ---- | ---- | ---- | ---- | ---- | ----- |
35
- | [WoonaV1.2-9b-imat-Q2_K.gguf](https://huggingface.co/secretmoon/WoonaV1.2-9b-GGUF-Imatrix/blob/main/WoonaV1.2-9b-imat-Q2_K.gguf) | Q2_K [imatrix] | 2 | 3.08 GB| 5.58 GB | very, significant quality loss - not recommended, but usable |
36
- | [WoonaV1.2-9b-imat-IQ3_XXS.gguf](https://huggingface.co/secretmoon/WoonaV1.2-9b-GGUF-Imatrix/blob/main/WoonaV1.2-9b-imat-IQ3_XXS.gguf) | Q3_K_S [imatrix] | 3 | 3.16 GB| 5.66 GB | small, high quality loss |
37
- | [WoonaV1.2-9b-imat-IQ3_M.gguf](https://huggingface.co/secretmoon/WoonaV1.2-9b-GGUF-Imatrix/blob/main/WoonaV1.2-9b-imat-IQ3_M.gguf) | IQ3_M [imatrix] | 3 | 3.52 GB| 6.02 GB | small, high quality loss |
38
- | [WoonaV1.2-9b-imat-IQ4_XS.gguf](https://huggingface.co/secretmoon/WoonaV1.2-9b-GGUF-Imatrix/blob/main/WoonaV1.2-9b-imat-IQ4_XS.gguf) | Q4_XS [imatrix] | 4 | 3.82 GB| 6.32 GB | medium, substantial quality loss |
39
- | [WoonaV1.2-9b-imat-Q4_K_S.gguf](https://huggingface.co/secretmoon/WoonaV1.2-9b-GGUF-Imatrix/blob/main/WoonaV1.2-9b-imat-Q4_K_S.gguf) | Q4_K_S [imatrix] | 4 | 4.14 GB| 6.64 GB | medium, balanced quality loss - recommended |
40
- | [WoonaV1.2-9b-imat-Q4_K_M.gguf](https://huggingface.co/secretmoon/WoonaV1.2-9b-GGUF-Imatrix/blob/main/WoonaV1.2-9b-imat-Q4_K_M.gguf) | Q4_K_M [imatrix] | 4 | 4.37 GB| 6.87 GB | medium, balanced quality - recommended |
41
- | [WoonaV1.2-9b-imat-Q5_K_S.gguf](https://huggingface.co/secretmoon/WoonaV1.2-9b-GGUF-Imatrix/blob/main/WoonaV1.2-9b-imat-Q5_K_S.gguf) | Q5_K_S [imatrix] | 5 | 5.00 GB| 7.50 GB | large, low quality loss - recommended |
42
- | [WoonaV1.2-9b-imat-Q5_K_M.gguf](https://huggingface.co/secretmoon/WoonaV1.2-9b-GGUF-Imatrix/blob/main/WoonaV1.2-9b-imat-Q5_K_M.gguf) | Q5_K_M [imatrix] | 5 | 5.13 GB| 7.63 GB | large, very low quality loss - recommended |
43
  | [WoonaV1.2-9b-Q6_K.gguf](https://huggingface.co/secretmoon/WoonaV1.2-9b-GGUF-Imatrix/blob/main/WoonaV1.2-9b-Q6_K.gguf) | Q6_K [static] | 6 | 7.1 GB| 8.7 GB | very large, near perfect loss - recommended |
44
  | [WoonaV1.2-9b-Q8_0.gguf](https://huggingface.co/secretmoon/WoonaV1.2-9b-GGUF-Imatrix/blob/main/WoonaV1.2-9b-Q6_K.gguf) | Q8_0 [static] | 8 | 9.2 GB| 10.8 GB | very large, extremely low quality loss
45
 
@@ -51,4 +51,4 @@ GGUF imatrix quants of **[AlexBefest/WoonaV1.2-9b](https://huggingface.co/AlexBe
51
  - **[koboldcpp](https://github.com/LostRuins/koboldcpp)**
52
  Easy method for windows inference. Lightweight open source fork llama.cpp with a simple graphical interface and many additional features.
53
  - **[LM studio](https://lmstudio.ai/)**
54
- Proprietary free fork llama.cpp with a graphical interface.
 
32
 
33
  | Name | Quant method | Bits | Size | Min RAM required | Use case |
34
  | ---- | ---- | ---- | ---- | ---- | ----- |
35
+ | [WoonaV1.2-9b-imat-Q2_K.gguf](https://huggingface.co/secretmoon/WoonaV1.2-9b-GGUF-Imatrix/blob/main/WoonaV1.2-9b-imat-Q2_K.gguf) | Q2_K [imatrix] | 2 | 3.5 GB| 5.1 GB | very, significant quality loss - not recommended, but usable (faster) |
36
+ | [WoonaV1.2-9b-imat-IQ3_XXS.gguf](https://huggingface.co/secretmoon/WoonaV1.2-9b-GGUF-Imatrix/blob/main/WoonaV1.2-9b-imat-IQ3_XXS.gguf) | IQ3_XXS [imatrix] | 3 | 3.5 GB| 5.1 GB | small, high quality loss |
37
+ | [WoonaV1.2-9b-imat-IQ3_M.gguf](https://huggingface.co/secretmoon/WoonaV1.2-9b-GGUF-Imatrix/blob/main/WoonaV1.2-9b-imat-IQ3_M.gguf) | IQ3_M [imatrix] | 3 | 4.2 GB| 5.7 GB | small, high quality loss |
38
+ | [WoonaV1.2-9b-imat-IQ4_XS.gguf](https://huggingface.co/secretmoon/WoonaV1.2-9b-GGUF-Imatrix/blob/main/WoonaV1.2-9b-imat-IQ4_XS.gguf) | Q4_XS [imatrix] | 4 | 4.8 GB| 6.3 GB | medium, substantial quality loss |
39
+ | [WoonaV1.2-9b-imat-Q4_K_S.gguf](https://huggingface.co/secretmoon/WoonaV1.2-9b-GGUF-Imatrix/blob/main/WoonaV1.2-9b-imat-Q4_K_S.gguf) | Q4_K_S [imatrix] | 4 | 5.1 GB| 6.7 GB | medium, balanced quality loss |
40
+ | [WoonaV1.2-9b-imat-Q4_K_M.gguf](https://huggingface.co/secretmoon/WoonaV1.2-9b-GGUF-Imatrix/blob/main/WoonaV1.2-9b-imat-Q4_K_M.gguf) | Q4_K_M [imatrix] | 4 | 5.4 GB| 6.9 GB | medium, balanced quality - recommended |
41
+ | [WoonaV1.2-9b-imat-Q5_K_S.gguf](https://huggingface.co/secretmoon/WoonaV1.2-9b-GGUF-Imatrix/blob/main/WoonaV1.2-9b-imat-Q5_K_S.gguf) | Q5_K_S [imatrix] | 5 | 6 GB| 7.6 GB | large, low quality loss - recommended |
42
+ | [WoonaV1.2-9b-imat-Q5_K_M.gguf](https://huggingface.co/secretmoon/WoonaV1.2-9b-GGUF-Imatrix/blob/main/WoonaV1.2-9b-imat-Q5_K_M.gguf) | Q5_K_M [imatrix] | 5 | 6.2 GB| 7.8 GB | large, very low quality loss - recommended |
43
  | [WoonaV1.2-9b-Q6_K.gguf](https://huggingface.co/secretmoon/WoonaV1.2-9b-GGUF-Imatrix/blob/main/WoonaV1.2-9b-Q6_K.gguf) | Q6_K [static] | 6 | 7.1 GB| 8.7 GB | very large, near perfect loss - recommended |
44
  | [WoonaV1.2-9b-Q8_0.gguf](https://huggingface.co/secretmoon/WoonaV1.2-9b-GGUF-Imatrix/blob/main/WoonaV1.2-9b-Q6_K.gguf) | Q8_0 [static] | 8 | 9.2 GB| 10.8 GB | very large, extremely low quality loss
45
 
 
51
  - **[koboldcpp](https://github.com/LostRuins/koboldcpp)**
52
  Easy method for windows inference. Lightweight open source fork llama.cpp with a simple graphical interface and many additional features.
53
  - **[LM studio](https://lmstudio.ai/)**
54
+ Proprietary free fork llama.cpp with a graphical interface.