量子化後のモデルのサイズと備考のまとめ

量子化後のモデルのサイズと備考のまとめです．どの量子化されたモデルを使うか迷ったとき，参考になるかも知れません．

モデルがGPUのメモリにのる割合が高いほど高速に動作します．モデルがGPUのメモリにすべてのるのが理想的です．量子化はビット数が多いほど精度が高いですが，大規模言語モデルの場合4ビット(q4)程度でも多くの場合使用に支障がないと言われています．

## Aya 8B

出典サイト
https://huggingface.co/bartowski/aya-23-8B-GGUF



|Filename| 	Quant type| 	File Size| 	Description|
|--|--|--|--|
|aya-23-8B-Q8_0.gguf 	|Q8_0   	|8.54GB 	|Extremely high quality, generally unneeded but max available quant.|
|aya-23-8B-Q6_K.gguf 	|Q6_K 	    |6.59GB 	|Very high quality, near perfect, recommended.|
|aya-23-8B-Q5_K_M.gguf 	|Q5_K_M 	|5.80GB 	|High quality, recommended.|
|aya-23-8B-Q5_K_S.gguf 	|Q5_K_S 	|5.66GB 	|High quality, recommended.|
|aya-23-8B-Q4_K_M.gguf 	|Q4_K_M 	|5.05GB 	|Good quality, uses about 4.83 bits per weight, recommended.|
|aya-23-8B-Q4_K_S.gguf 	|Q4_K_S 	|4.82GB 	|Slightly lower quality with more space savings, recommended.|
|aya-23-8B-IQ4_NL.gguf 	|IQ4_NL 	|4.81GB 	|Decent quality, slightly smaller than Q4_K_S with similar performance recommended.|
|aya-23-8B-IQ4_XS.gguf 	|IQ4_XS 	|4.60GB 	|Decent quality, smaller than Q4_K_S with similar performance, recommended.|
|aya-23-8B-Q3_K_L.gguf 	|Q3_K_L 	|4.52GB 	|Lower quality but usable, good for low RAM availability.|
|aya-23-8B-Q3_K_M.gguf 	|Q3_K_M 	|4.22GB 	|Even lower quality.|
|aya-23-8B-IQ3_M.gguf 	|IQ3_M 	    |3.99GB 	|Medium-low quality, new method with decent performance comparable to Q3_K_M.|
|aya-23-8B-IQ3_S.gguf 	|IQ3_S 	    |3.88GB 	|Lower quality, new method with decent performance, recommended over Q3_K_S quant, same size with better performance.|
|aya-23-8B-Q3_K_S.gguf 	|Q3_K_S 	|3.87GB 	|Low quality, not recommended.|
|aya-23-8B-IQ3_XS.gguf 	|IQ3_XS 	|3.72GB 	|Lower quality, new method with decent performance, slightly better than Q3_K_S.|
|aya-23-8B-IQ3_XXS.gguf |IQ3_XXS 	|3.41GB 	|Lower quality, new method with decent performance, comparable to Q3 quants.|
|aya-23-8B-Q2_K.gguf 	|Q2_K 	    |3.43GB 	|Very low quality but surprisingly usable.|
|aya-23-8B-IQ2_M.gguf 	|IQ2_M 	    |3.08GB 	|Very low quality, uses SOTA techniques to also be surprisingly usable.|
|aya-23-8B-IQ2_S.gguf 	|IQ2_S 	    |2.89GB 	|Very low quality, uses SOTA techniques to be usable.|
|aya-23-8B-IQ2_XS.gguf 	|IQ2_XS 	|2.79GB 	|Very low quality, uses SOTA techniques to be usable.|
|aya-23-8B-IQ2_XXS.gguf	|IQ2_XXS 	|2.58GB 	|Lower quality, uses SOTA techniques to be usable.|
|aya-23-8B-IQ1_M.gguf 	|IQ1_M 	    |2.35GB 	|Extremely low quality, not recommended.|
|aya-23-8B-IQ1_S.gguf 	|IQ1_S 	    |2.20GB 	|Extremely low quality, not recommended.Filename 	Quant type 	File Size 	Description|



## Llama3 8B

出典サイト
https://huggingface.co/bartowski/Meta-Llama-3-8B-Instruct-GGUF

|Filename| 	Quant type| 	File Size| 	Description|
|--|--|--|--|
|Meta-Llama-3-8B-Instruct-Q8_0.gguf 	|Q8_0 	    |8.54GB 	|Extremely high quality, generally unneeded but max available quant.|
|Meta-Llama-3-8B-Instruct-Q6_K.gguf 	|Q6_K 	    |6.59GB 	|Very high quality, near perfect, recommended.|
|Meta-Llama-3-8B-Instruct-Q5_K_M.gguf 	|Q5_K_M 	|5.73GB 	|High quality, recommended.|
|Meta-Llama-3-8B-Instruct-Q5_K_S.gguf 	|Q5_K_S 	|5.59GB 	|High quality, recommended.|
|Meta-Llama-3-8B-Instruct-Q4_K_M.gguf 	|Q4_K_M 	|4.92GB 	|Good quality, uses about 4.83 bits per weight, recommended.|
|Meta-Llama-3-8B-Instruct-Q4_K_S.gguf 	|Q4_K_S 	|4.69GB 	|Slightly lower quality with more space savings, recommended.|
|Meta-Llama-3-8B-Instruct-IQ4_NL.gguf 	|IQ4_NL 	|4.67GB 	|Decent quality, slightly smaller than Q4_K_S with similar performance recommended.|
|Meta-Llama-3-8B-Instruct-IQ4_XS.gguf 	|IQ4_XS 	|4.44GB 	|Decent quality, smaller than Q4_K_S with similar performance, recommended.|
|Meta-Llama-3-8B-Instruct-Q3_K_L.gguf 	|Q3_K_L 	|4.32GB 	|Lower quality but usable, good for low RAM availability.|
|Meta-Llama-3-8B-Instruct-Q3_K_M.gguf 	|Q3_K_M 	|4.01GB 	|Even lower quality.|
|Meta-Llama-3-8B-Instruct-IQ3_M.gguf 	|IQ3_M 	    |3.78GB 	|Medium-low quality, new method with decent performance comparable to Q3_K_M.|
|Meta-Llama-3-8B-Instruct-IQ3_S.gguf 	|IQ3_S    	|3.68GB 	|Lower quality, new method with decent performance, recommended over Q3_K_S quant, same size with better performance.|
|Meta-Llama-3-8B-Instruct-Q3_K_S.gguf 	|Q3_K_S 	|3.66GB 	|Low quality, not recommended.|
|Meta-Llama-3-8B-Instruct-IQ3_XS.gguf 	|IQ3_XS 	|3.51GB 	|Lower quality, new method with decent performance, slightly better than Q3_K_S.|
|Meta-Llama-3-8B-Instruct-IQ3_XXS.gguf 	|IQ3_XXS 	|3.27GB 	|Lower quality, new method with decent performance, comparable to Q3 quants.|
|Meta-Llama-3-8B-Instruct-Q2_K.gguf 	|Q2_K 	    |3.17GB 	|Very low quality but surprisingly usable.|
|Meta-Llama-3-8B-Instruct-IQ2_M.gguf 	|IQ2_M 	    |2.94GB 	|Very low quality, uses SOTA techniques to also be surprisingly usable.|
|Meta-Llama-3-8B-Instruct-IQ2_S.gguf 	|IQ2_S 	    |2.75GB 	|Very low quality, uses SOTA techniques to be usable.|
|Meta-Llama-3-8B-Instruct-IQ2_XS.gguf 	|IQ2_XS 	|2.60GB 	|Very low quality, uses SOTA techniques to be usable.|
|Meta-Llama-3-8B-Instruct-IQ2_XXS.gguf 	|IQ2_XXS 	|2.39GB 	|Lower quality, uses SOTA techniques to be usable.|
|Meta-Llama-3-8B-Instruct-IQ1_M.gguf 	|IQ1_M 	    |2.16GB 	|Extremely low quality, not recommended.|
|Meta-Llama-3-8B-Instruct-IQ1_S.gguf 	|IQ1_S 	    |2.01GB 	|Extremely low quality, not recommended.|

## Llama3 70b

出典サイト
https://huggingface.co/bartowski/Meta-Llama-3-70B-Instruct-old-GGUF

|Filename| 	Quant type| 	File Size| 	Description|
|--|--|--|--|
|Meta-Llama-3-70B-Instruct-Q8_0.gguf 	 |Q8_0 	    |74.97GB 	|Extremely high quality, generally unneeded but max available quant.|
|Meta-Llama-3-70B-Instruct-Q6_K.gguf 	 |Q6_K   	|57.88GB 	|Very high quality, near perfect, recommended.|
|Meta-Llama-3-70B-Instruct-Q5_K_M.gguf 	 |Q5_K_M 	|49.94GB 	|High quality, recommended.|
|Meta-Llama-3-70B-Instruct-Q5_K_S.gguf 	 |Q5_K_S 	|48.65GB 	|High quality, recommended.|
|Meta-Llama-3-70B-Instruct-Q4_K_M.gguf 	 |Q4_K_M 	|42.52GB 	|Good quality, uses about 4.83 bits per weight, recommended.|
|Meta-Llama-3-70B-Instruct-Q4_K_S.gguf 	 |Q4_K_S 	|40.34GB 	|Slightly lower quality with more space savings, recommended.|
|Meta-Llama-3-70B-Instruct-IQ4_NL.gguf 	 |IQ4_NL 	|40.05GB 	|Decent quality, slightly smaller than Q4_K_S with similar performance recommended.|
|Meta-Llama-3-70B-Instruct-IQ4_XS.gguf 	 |IQ4_XS 	|37.90GB 	|Decent quality, smaller than Q4_K_S with similar performance, recommended.|
|Meta-Llama-3-70B-Instruct-Q3_K_L.gguf 	 |Q3_K_L 	|37.14GB 	|Lower quality but usable, good for low RAM availability.|
|Meta-Llama-3-70B-Instruct-Q3_K_M.gguf 	 |Q3_K_M 	|34.26GB 	|Even lower quality.|
|Meta-Llama-3-70B-Instruct-IQ3_M.gguf 	 |IQ3_M 	|31.93GB 	|Medium-low quality, new method with decent performance comparable to Q3_K_M.|
|Meta-Llama-3-70B-Instruct-IQ3_S.gguf 	 |IQ3_S 	|30.91GB 	|Lower quality, new method with decent performance, recommended over Q3_K_S quant, same size with better performance.|
|Meta-Llama-3-70B-Instruct-Q3_K_S.gguf 	 |Q3_K_S 	|30.91GB 	|Low quality, not recommended.|
|Meta-Llama-3-70B-Instruct-IQ3_XS.gguf 	 |IQ3_XS 	|29.30GB 	|Lower quality, new method with decent performance, slightly better than Q3_K_S.|
|Meta-Llama-3-70B-Instruct-IQ3_XXS.gguf  |IQ3_XXS 	|27.46GB 	|Lower quality, new method with decent performance, comparable to Q3 quants.|
|Meta-Llama-3-70B-Instruct-Q2_K.gguf 	 |Q2_K 	    |26.37GB 	|Very low quality but surprisingly usable.|
|Meta-Llama-3-70B-Instruct-IQ2_M.gguf 	 |IQ2_M 	|24.11GB 	|Very low quality, uses SOTA techniques to also be surprisingly usable.|
|Meta-Llama-3-70B-Instruct-IQ2_S.gguf 	 |IQ2_S 	|22.24GB 	|Very low quality, uses SOTA techniques to be usable.|
|Meta-Llama-3-70B-Instruct-IQ2_XS.gguf 	 |IQ2_XS 	|21.14GB 	|Very low quality, uses SOTA techniques to be usable.|
|Meta-Llama-3-70B-Instruct-IQ2_XXS.gguf  |IQ2_XXS 	|19.09GB 	|Lower quality, uses SOTA techniques to be usable.|
|Meta-Llama-3-70B-Instruct-IQ1_M.gguf 	 |IQ1_M 	|16.75GB 	|Extremely low quality, not recommended.|
|Meta-Llama-3-70B-Instruct-IQ1_S.gguf 	 |IQ1_S 	|15.34GB 	|Extremely low quality, not recommended.|

## Phi3-medium

出典サイト
https://huggingface.co/bartowski/Phi-3-medium-4k-instruct-GGUF

|Filename| 	Quant type| 	File Size| 	Description|
|--|--|--|--|
|Phi-3-medium-4k-instruct-Q8_0.gguf 	|Q8_0 	    |14.83GB| 	Extremely high quality, generally unneeded but max available quant.|
|Phi-3-medium-4k-instruct-Q6_K.gguf 	|Q6_K   	|11.45GB| 	Very high quality, near perfect, recommended.|
|Phi-3-medium-4k-instruct-Q5_K_M.gguf 	|Q5_K_M 	|10.07GB| 	High quality, recommended.|
|Phi-3-medium-4k-instruct-Q5_K_S.gguf 	|Q5_K_S 	|9.62GB |	High quality, recommended.|
|Phi-3-medium-4k-instruct-Q4_K_M.gguf 	|Q4_K_M 	|8.56GB |	Good quality, uses about 4.83 bits per weight, recommended.|
|Phi-3-medium-4k-instruct-Q4_K_S.gguf 	|Q4_K_S 	|7.95GB |	Slightly lower quality with more space savings, recommended.|
|Phi-3-medium-4k-instruct-IQ4_NL.gguf 	|IQ4_NL 	|7.89GB |	Decent quality, slightly smaller than Q4_K_S with similar performance recommended.|
|Phi-3-medium-4k-instruct-IQ4_XS.gguf 	|IQ4_XS 	|7.46GB |	Decent quality, smaller than Q4_K_S with similar performance, recommended.|
|Phi-3-medium-4k-instruct-Q3_K_L.gguf 	|Q3_K_L 	|7.49GB |	Lower quality but usable, good for low RAM availability.|
|Phi-3-medium-4k-instruct-Q3_K_M.gguf 	|Q3_K_M 	|6.92GB |	Even lower quality.|
|Phi-3-medium-4k-instruct-IQ3_M.gguf 	|IQ3_M  	|6.47GB |	Medium-low quality, new method with decent performance comparable to Q3_K_M.|
|Phi-3-medium-4k-instruct-IQ3_S.gguf 	|IQ3_S 	    |6.06GB |	Lower quality, new method with decent performance, recommended over Q3_K_S quant, same size with better performance.|
|Phi-3-medium-4k-instruct-Q3_K_S.gguf 	|Q3_K_S 	|6.06GB |	Low quality, not recommended.|
|Phi-3-medium-4k-instruct-IQ3_XS.gguf 	|IQ3_XS 	|5.80GB |	Lower quality, new method with decent performance, slightly better than Q3_K_S.|
|Phi-3-medium-4k-instruct-IQ3_XXS.gguf 	|IQ3_XXS 	|5.45GB |	Lower quality, new method with decent performance, comparable to Q3 quants.|
|Phi-3-medium-4k-instruct-Q2_K.gguf 	|Q2_K   	|5.14GB |	Very low quality but surprisingly usable.|
|Phi-3-medium-4k-instruct-IQ2_M.gguf 	|IQ2_M  	|4.71GB |	Very low quality, uses SOTA techniques to also be surprisingly usable.|
|Phi-3-medium-4k-instruct-IQ2_S.gguf 	|IQ2_S 	    |4.33GB |	Very low quality, uses SOTA techniques to be usable.|
|Phi-3-medium-4k-instruct-IQ2_XS.gguf 	|IQ2_XS 	|4.12GB |	Very low quality, uses SOTA techniques to be usable.|
|Phi-3-medium-4k-instruct-IQ2_XXS.gguf 	|IQ2_XXS 	|3.71GB |	Lower quality, uses SOTA techniques to be usable.|
|Phi-3-medium-4k-instruct-IQ1_M.gguf 	|IQ1_M 	    |3.24GB |	Extremely low quality, not recommended.|
|Phi-3-medium-4k-instruct-IQ1_S.gguf 	|IQ1_S    	|2.95GB |	Extremely low quality, not recommended.|

## Mixtral 8x7b

出典サイト
https://huggingface.co/TheBloke/Mixtral-8x7B-Instruct-v0.1-GGUF

|Name| 	Quant method| 	Bits| 	Size| 	Max RAM required| 	Use case|
|--|--|--|--|--|--|
|mixtral-8x7b-instruct-v0.1.Q2_K.gguf 	    |Q2_K 	    |2 	|15.64 GB 	|18.14 GB 	|smallest, significant quality loss - not recommended for most purposes|
|mixtral-8x7b-instruct-v0.1.Q3_K_M.gguf 	|Q3_K_M 	|3 	|20.36 GB 	|22.86 GB 	|very small, high quality loss|
|mixtral-8x7b-instruct-v0.1.Q4_0.gguf 	    |Q4_0 	    |4 	|26.44 GB 	|28.94 GB 	|legacy; small, very high quality loss - prefer using Q3_K_M|
|mixtral-8x7b-instruct-v0.1.Q4_K_M.gguf 	|Q4_K_M 	|4 	|26.44 GB 	|28.94 GB 	|medium, balanced quality - recommended|
|mixtral-8x7b-instruct-v0.1.Q5_0.gguf 	    |Q5_0 	    |5 	|32.23 GB 	|34.73 GB 	|legacy; medium, balanced quality - prefer using Q4_K_M|
|mixtral-8x7b-instruct-v0.1.Q5_K_M.gguf 	|Q5_K_M 	|5 	|32.23 GB 	|34.73 GB 	|large, very low quality loss - recommended|
|mixtral-8x7b-instruct-v0.1.Q6_K.gguf 	    |Q6_K 	    |6 	|38.38 GB 	|40.88 GB 	|very large, extremely low quality loss|
|mixtral-8x7b-instruct-v0.1.Q8_0.gguf 	    |Q8_0 	    |8 	|49.62 GB 	|52.12 GB 	|very large, extremely low quality loss - not recommended|

## Starling-LM 7b

出典サイト
https://huggingface.co/bartowski/Starling-LM-7B-beta-GGUF

|Name| 	Quant method|  	Size|  	Use case|
|--|--|--|--|
|Starling-LM-7B-beta-Q8_0.gguf 	    |Q8_0     	|7.69GB 	|Extremely high quality, generally unneeded but max available quant.|
|Starling-LM-7B-beta-Q6_K.gguf 	    |Q6_K 	    |5.94GB 	|Very high quality, near perfect, recommended.|
|Starling-LM-7B-beta-Q5_K_M.gguf 	|Q5_K_M 	|5.13GB 	|High quality, very usable.|
|Starling-LM-7B-beta-Q5_K_S.gguf 	|Q5_K_S 	|4.99GB 	|High quality, very usable.|
|Starling-LM-7B-beta-Q5_0.gguf 	    |Q5_0 	    |4.99GB 	|High quality, older format, generally not recommended.|
|Starling-LM-7B-beta-Q4_K_M.gguf 	|Q4_K_M 	|4.36GB 	|Good quality, similar to 4.25 bpw.|
|Starling-LM-7B-beta-Q4_K_S.gguf 	|Q4_K_S 	|4.14GB 	|Slightly lower quality with small space savings.|
|Starling-LM-7B-beta-IQ4_NL.gguf 	|IQ4_NL 	|4.15GB 	|Good quality, similar to Q4_K_S, new method of quanting,|
|Starling-LM-7B-beta-IQ4_XS.gguf 	|IQ4_XS 	|3.94GB 	|Decent quality, new method with similar performance to Q4.|
|Starling-LM-7B-beta-Q4_0.gguf 	    |Q4_0 	    |4.10GB 	|Decent quality, older format, generally not recommended.|
|Starling-LM-7B-beta-IQ3_M.gguf 	|IQ3_M 	    |3.28GB 	|Medium-low quality, new method with decent performance.|
|Starling-LM-7B-beta-IQ3_S.gguf 	|IQ3_S 	    |3.18GB 	|Lower quality, new method with decent performance, recommended over Q3 quants.|
|Starling-LM-7B-beta-Q3_K_L.gguf 	|Q3_K_L 	|3.82GB 	|Lower quality but usable, good for low RAM availability.|
|Starling-LM-7B-beta-Q3_K_M.gguf 	|Q3_K_M 	|3.51GB 	|Even lower quality.|
|Starling-LM-7B-beta-Q3_K_S.gguf 	|Q3_K_S 	|3.16GB 	|Low quality, not recommended.|
|Starling-LM-7B-beta-Q2_K.gguf 	    |Q2_K 	    |2.71GB 	|Extremely low quality, not recommended.|

## WizardLM2 7b

出典サイト
https://huggingface.co/bartowski/WizardLM-2-7B-GGUF


|Filename| 	Quant type| 	File Size| 	Description|
|--|--|--|--|
|WizardLM-2-7B-Q8_0.gguf 	    |Q8_0 	    |7.69GB 	|Extremely high quality, generally unneeded but max available quant.|
|WizardLM-2-7B-Q6_K.gguf 	    |Q6_K 	    |5.94GB 	|Very high quality, near perfect, recommended.|
|WizardLM-2-7B-Q5_K_M.gguf 	    |Q5_K_M 	|5.13GB 	|High quality, recommended.|
|WizardLM-2-7B-Q5_K_S.gguf 	    |Q5_K_S 	|4.99GB 	|High quality, recommended.|
|WizardLM-2-7B-Q4_K_M.gguf 	    |Q4_K_M 	|4.36GB 	|Good quality, uses about 4.83 bits per weight, recommended.|
|WizardLM-2-7B-Q4_K_S.gguf 	    |Q4_K_S 	|4.14GB 	|Slightly lower quality with more space savings, recommended.|
|WizardLM-2-7B-IQ4_NL.gguf 	    |IQ4_NL 	|4.12GB 	|Decent quality, slightly smaller than Q4_K_S with similar performance recommended.|
|WizardLM-2-7B-IQ4_XS.gguf 	    |IQ4_XS 	|3.90GB 	|Decent quality, smaller than Q4_K_S with similar performance, recommended.|
|WizardLM-2-7B-Q3_K_L.gguf 	    |Q3_K_L 	|3.82GB 	|Lower quality but usable, good for low RAM availability.|
|WizardLM-2-7B-Q3_K_M.gguf 	    |Q3_K_M 	|3.51GB 	|Even lower quality.|
|WizardLM-2-7B-IQ3_M.gguf 	    |IQ3_M 	    |3.28GB 	|Medium-low quality, new method with decent performance comparable to Q3_K_M.|
|WizardLM-2-7B-IQ3_S.gguf 	    |IQ3_S 	    |3.18GB 	|Lower quality, new method with decent performance, recommended over Q3_K_S quant, same size with better performance.|
|WizardLM-2-7B-Q3_K_S.gguf 	    |Q3_K_S 	|3.16GB 	|Low quality, not recommended.|
|WizardLM-2-7B-IQ3_XS.gguf 	    |IQ3_XS 	|3.01GB 	|Lower quality, new method with decent performance, slightly better than Q3_K_S.|
|WizardLM-2-7B-IQ3_XXS.gguf 	|IQ3_XXS 	|2.82GB 	|Lower quality, new method with decent performance, comparable to Q3 quants.|
|WizardLM-2-7B-Q2_K.gguf 	    |Q2_K 	    |2.71GB 	|Very low quality but surprisingly usable.|
|WizardLM-2-7B-IQ2_M.gguf 	    |IQ2_M 	    |2.50GB 	|Very low quality, uses SOTA techniques to also be surprisingly usable.|
|WizardLM-2-7B-IQ2_S.gguf 	    |IQ2_S 	    |2.31GB 	|Very low quality, uses SOTA techniques to be usable.|
|WizardLM-2-7B-IQ2_XS.gguf 	    |IQ2_XS 	|2.19GB 	|Very low quality, uses SOTA techniques to be usable.|
|WizardLM-2-7B-IQ2_XXS.gguf 	|IQ2_XXS 	|1.99GB 	|Lower quality, uses SOTA techniques to be usable.|
|WizardLM-2-7B-IQ1_M.gguf 	    |IQ1_M 	    |1.75GB 	|Extremely low quality, not recommended.|
|WizardLM-2-7B-IQ1_S.gguf 	    |IQ1_S 	    |1.61GB 	|Extremely low quality, not recommended.|

## Codestral 22B

出典サイト
https://huggingface.co/bartowski/Codestral-22B-v0.1-GGUF


|Filename| 	Quant type| 	File Size| 	Description|
|--|--|--|--|
|Codestral-22B-v0.1-Q8_0.gguf   	|Q8_0   	|23.64GB| 	Extremely high quality, generally unneeded but max available quant.|
|Codestral-22B-v0.1-Q6_K.gguf   	|Q6_K   	|18.25GB| 	Very high quality, near perfect, recommended.|
|Codestral-22B-v0.1-Q5_K_M.gguf 	|Q5_K_M 	|15.72GB| 	High quality, recommended.|
|Codestral-22B-v0.1-Q5_K_S.gguf 	|Q5_K_S 	|15.32GB| 	High quality, recommended.|
|Codestral-22B-v0.1-Q4_K_M.gguf 	|Q4_K_M 	|13.34GB| 	Good quality, uses about 4.83 bits per weight, recommended.|
|Codestral-22B-v0.1-Q4_K_S.gguf 	|Q4_K_S 	|12.66GB| 	Slightly lower quality with more space savings, recommended.|
|Codestral-22B-v0.1-IQ4_XS.gguf 	|IQ4_XS 	|11.93GB| 	Decent quality, smaller than Q4_K_S with similar performance, recommended.|
|Codestral-22B-v0.1-Q3_K_L.gguf 	|Q3_K_L 	|11.73GB| 	Lower quality but usable, good for low RAM availability.|
|Codestral-22B-v0.1-Q3_K_M.gguf 	|Q3_K_M 	|10.75GB| 	Even lower quality.|
|Codestral-22B-v0.1-IQ3_M.gguf  	|IQ3_M  	|10.06GB| 	Medium-low quality, new method with decent performance comparable to Q3_K_M.|
|Codestral-22B-v0.1-Q3_K_S.gguf 	|Q3_K_S 	|9.64GB |	Low quality, not recommended.|
|Codestral-22B-v0.1-IQ3_XS.gguf 	|IQ3_XS 	|9.17GB |	Lower quality, new method with decent performance, slightly better than Q3_K_S.|
|Codestral-22B-v0.1-IQ3_XXS.gguf 	|IQ3_XXS 	|8.59GB |	Lower quality, new method with decent performance, comparable to Q3 quants.|
|Codestral-22B-v0.1-Q2_K.gguf   	|Q2_K   	|8.27GB |	Very low quality but surprisingly usable.|
|Codestral-22B-v0.1-IQ2_M.gguf  	|IQ2_M  	|7.61GB |	Very low quality, uses SOTA techniques to also be surprisingly usable.|
|Codestral-22B-v0.1-IQ2_S.gguf  	|IQ2_S  	|7.03GB |	Very low quality, uses SOTA techniques to be usable.|
|Codestral-22B-v0.1-IQ2_XS.gguf 	|IQ2_XS 	|6.64GB |	Very low quality, uses SOTA techniques to be usable.|


## Deepseek coder 33b

出典サイト
https://huggingface.co/TheBloke/deepseek-coder-33B-base-GGUF

|Name                            	    |Quant method 	|Bits 	|Size 	    |Max RAM  required  | 	Use case|
|--|--|--|--|--|--|
|deepseek-coder-33b-base.Q2_K.gguf 	    |Q2_K    	    |2      |14.03 GB 	|16.53 GB 	        |smallest, significant quality loss - not recommended for most purposes|
|deepseek-coder-33b-base.Q3_K_S.gguf 	|Q3_K_S 	    |3      |14.42 GB 	|16.92 GB 	        |very small, high quality loss|
|deepseek-coder-33b-base.Q3_K_M.gguf 	|Q3_K_M 	    |3      |16.07 GB 	|18.57 GB 	        |very small, high quality loss|
|deepseek-coder-33b-base.Q3_K_L.gguf 	|Q3_K_L 	    |3      |17.56 GB 	|20.06 GB 	        |small, substantial quality loss|
|deepseek-coder-33b-base.Q4_0.gguf 	    |Q4_0 	        |4      |18.82 GB 	|21.32 GB 	        |legacy; small, very high quality loss - prefer using Q3_K_M|
|deepseek-coder-33b-base.Q4_K_S.gguf 	|Q4_K_S 	    |4      |18.89 GB 	|21.39 GB 	        |small, greater quality loss|
|deepseek-coder-33b-base.Q4_K_M.gguf 	|Q4_K_M 	    |4      |19.94 GB 	|22.44 GB 	        |medium, balanced quality - recommended|
|deepseek-coder-33b-base.Q5_0.gguf    	|Q5_0 	        |5      |22.96 GB 	|25.46 GB 	        |legacy; medium, balanced quality - prefer using Q4_K_M|
|deepseek-coder-33b-base.Q5_K_S.gguf 	|Q5_K_S 	    |5      |22.96 GB 	|25.46 GB 	        |large, low quality loss - recommended|
|deepseek-coder-33b-base.Q5_K_M.gguf 	|Q5_K_M 	    |5      |23.54 GB 	|26.04 GB 	        |large, very low quality loss - recommended|
|deepseek-coder-33b-base.Q6_K.gguf 	    |Q6_K 	        |6      |27.36 GB 	|29.86 GB 	        |very large, extremely low quality loss|
|deepseek-coder-33b-base.Q8_0.gguf 	    |Q8_0 	        |8      |35.43 GB 	|37.93 GB 	        |very large, extremely low quality loss - not recommended|

## Deepseek coder 6.7b

出典サイト
https://huggingface.co/TheBloke/deepseek-coder-6.7B-instruct-GGUF

|Name                            	    |Quant method 	|Bits 	|Size 	    |Max RAM  required  | 	Use case|
|--|--|--|--|--|--|
|deepseek-coder-6.7b-instruct.Q2_K.gguf 	|Q2_K 	    |2 	|2.83 GB 	|5.33 GB 	|smallest, significant quality loss - not recommended for most purposes|
|deepseek-coder-6.7b-instruct.Q3_K_S.gguf 	|Q3_K_S 	|3 	|2.95 GB 	|5.45 GB 	|very small, high quality loss|
|deepseek-coder-6.7b-instruct.Q3_K_M.gguf 	|Q3_K_M 	|3 	|3.30 GB 	|5.80 GB 	|very small, high quality loss|
|deepseek-coder-6.7b-instruct.Q3_K_L.gguf 	|Q3_K_L 	|3 	|3.60 GB 	|6.10 GB 	|small, substantial quality loss|
|deepseek-coder-6.7b-instruct.Q4_0.gguf 	|Q4_0 	    |4 	|3.83 GB 	|6.33 GB 	|legacy; small, very high quality loss - prefer using Q3_K_M|
|deepseek-coder-6.7b-instruct.Q4_K_S.gguf 	|Q4_K_S 	|4 	|3.86 GB 	|6.36 GB 	|small, greater quality loss|
|deepseek-coder-6.7b-instruct.Q4_K_M.gguf 	|Q4_K_M 	|4 	|4.08 GB 	|6.58 GB 	|medium, balanced quality - recommended|
|deepseek-coder-6.7b-instruct.Q5_0.gguf 	|Q5_0 	    |5 	|4.65 GB 	|7.15 GB 	|legacy; medium, balanced quality - prefer using Q4_K_M|
|deepseek-coder-6.7b-instruct.Q5_K_S.gguf 	|Q5_K_S 	|5 	|4.65 GB 	|7.15 GB 	|large, low quality loss - recommended|
|deepseek-coder-6.7b-instruct.Q5_K_M.gguf 	|Q5_K_M 	|5 	|4.79 GB 	|7.29 GB 	|large, very low quality loss - recommended|
|deepseek-coder-6.7b-instruct.Q6_K.gguf 	|Q6_K 	    |6 	|5.53 GB 	|8.03 GB 	|very large, extremely low quality loss|
|deepseek-coder-6.7b-instruct.Q8_0.gguf 	|Q8_0 	    |8 	|7.16 GB 	|9.66 GB 	|very large, extremely low quality loss - not recommended|

## wizardcoder 33b

出典サイト
https://huggingface.co/TheBloke/WizardCoder-33B-V1.1-GGUF

|Name                            	    |Quant method 	|Bits 	|Size 	    |Max RAM  required  | 	Use case|
|--|--|--|--|--|--|
|wizardcoder-33b-v1.1.Q2_K.gguf 	|Q2_K     	|2 	|14.03 GB 	|16.53 GB 	|smallest, significant quality loss - not recommended for most purposes|
|wizardcoder-33b-v1.1.Q3_K_S.gguf 	|Q3_K_S 	|3 	|14.42 GB 	|16.92 GB 	|very small, high quality loss|
|wizardcoder-33b-v1.1.Q3_K_M.gguf 	|Q3_K_M 	|3 	|16.07 GB 	|18.57 GB 	|very small, high quality loss|
|wizardcoder-33b-v1.1.Q3_K_L.gguf 	|Q3_K_L 	|3 	|17.56 GB 	|20.06 GB 	|small, substantial quality loss|
|wizardcoder-33b-v1.1.Q4_0.gguf 	|Q4_0 	    |4 	|18.82 GB 	|21.32 GB 	|legacy; small, very high quality loss - prefer using Q3_K_M|
|wizardcoder-33b-v1.1.Q4_K_S.gguf 	|Q4_K_S 	|4 	|18.89 GB 	|21.39 GB 	|small, greater quality loss|
|wizardcoder-33b-v1.1.Q4_K_M.gguf 	|Q4_K_M 	|4 	|19.94 GB 	|22.44 GB 	|medium, balanced quality - recommended|
|wizardcoder-33b-v1.1.Q5_0.gguf 	|Q5_0 	    |5 	|22.96 GB 	|25.46 GB 	|legacy; medium, balanced quality - prefer using Q4_K_M|
|wizardcoder-33b-v1.1.Q5_K_S.gguf 	|Q5_K_S 	|5 	|22.96 GB 	|25.46 GB 	|large, low quality loss - recommended|
|wizardcoder-33b-v1.1.Q5_K_M.gguf 	|Q5_K_M 	|5 	|23.54 GB 	|26.04 GB 	|large, very low quality loss - recommended|
|wizardcoder-33b-v1.1.Q6_K.gguf 	|Q6_K    	|6 	|27.36 GB 	|29.86 GB 	|very large, extremely low quality loss|
|wizardcoder-33b-v1.1.Q8_0.gguf 	|Q8_0 	    |8 	|35.43 GB 	|37.93 GB 	|very large, extremely low quality loss - not recommended|