INT8 Quants for google-gemma-3-27b-it
#1
by
dazipe
- opened
I see you use AMD MI210. I have 2 x MI100 and I could not find any quants which work for me in vLLM.
They are all unsupported on MI100.
May I ask you to quantize it in INT8?
If not, would you please share your quantization script?