niallturbitt/mpt-3b-8k-instruct-GGUF

Quantized GGUF model files for mpt-3b-8k-instruct from niallturbitt

Name	Quant method	Size
mpt-3b-8k-instruct.q2_k.gguf	q2_k	1.54 GB
mpt-3b-8k-instruct.q3_k_m.gguf	q3_k_m	1.95 GB
mpt-3b-8k-instruct.q4_k_m.gguf	q4_k_m	2.34 GB
mpt-3b-8k-instruct.q5_k_m.gguf	q5_k_m	2.71 GB
mpt-3b-8k-instruct.q6_k.gguf	q6_k	2.98 GB
mpt-3b-8k-instruct.q8_0.gguf	q8_0	3.86 GB

Original Model Card: