GPTQ Quants
#11
by
rjmehta
- opened
Vllm supports gptq. Please gptq quants. @LoneStriker @MaziyarPanahi
@rjmehta
I don't need one of my A100, so I'll make a GPTQ of this model now.
it's available here: MaziyarPanahi/Truthful_DPO_TomGrc_FusionNet_7Bx2_MoE_13B-GPTQ