wtdcode

/

GLM-5.3-Flash-AWQ-W4A16

Like 22

Hi,
first thanks a lot for this quantisation....
i see on the model page that you left many things in bf16 and the smaller quants to int4 g128...

but would you mind sharing the exact setup you used to create those and the script to create the exact weights? i would realy like to reproduce it and never created a AWQ quant that works so well with a100's

i think you where using llm-compressor, maybe if you have 5 minutes or so and would share the exact script (or a minified version in case it has some private content of yours inside)