Need help with quantizing the original model

#1
by aaryaman - opened

Can someone provide instructions on how the original model can be quantized?
I downloaded the model from microsoft/phi-2 and tried to quantize it using the scripts in llama.cpp but got an error only to realize the model is not yet supported on llama.cpp.

Any insights or suggestions would be greatly appreciated.

I think python code is used maybe python transformers library or the one used in example code

@aaryaman were you able to quantize phi-2?

Sign up or log in to comment