Quantisation question?
#5
by
pjw000
- opened
Using this model should I be able to create a GPTQ 4 bit version? My naive attempts using the official intruct model produce the following error which I have no idea how to deal with:
ValueError: Block pattern could not be match. Pass block_name_to_quantize argument in quantize_model
Specifically: I have no idea what the block name might be!
There are already people trying to get a GPTQ version. I'd suggest to have a look at this github discussion: https://github.com/AutoGPTQ/AutoGPTQ/issues/621
Thank you. That's a VERY helpful link! Much appreciated.
pjw000
changed discussion status to
closed