AWQ quantization (on volta..)
#1
by cryowastakenbycryo - opened
Thanks for breaking down what is quantized and what you skipped. I've been fighting w/ getting AWQ quantization to work for a while now. I'm working w/ volta, w/o support from newer libraries so it's been a lot of trial and error. Any tips are appreciated.
Sorry I do not have access to such hardware.