Model Quantization Help
#1
by
HirCoir
- opened
Model Quantization Help
Can you tell me how you were able to quantize the Qwen2 model? I have tried with llama.cpp but once quantized it does not work, it gives an error when loading the model! I want to quantize a model that I have fine-tuned and runs under qwen2 0.5b
Without the error, it's hard to know what could have been gone wrong. I didn't do anything special for this model.