problem with index
#4
by
prudant
- opened
the name of the safetensor index did not work for quant the model, it has to be renamed to model.safetensors.index.json
I cloned your repo and tested in local and with that rename it worked.
Will try to get an AWQ quant (its more quick tan gptq) , I'm Dario from discord, let you know any news.
Note that the files are .bin — not .safetensor
burp, thats make all the sense of my problems, any way the quant is on the oven now
prudant
changed discussion status to
closed