RuntimeError: weight gptq_bits does not exist TGI
#22
by
MrAiran
- opened
I would like some help, I believe there must be a way to load the model via TGI, but whenever I try it returns the following error:
RuntimeError: weight gptq_bits does not exist
is there any way to load the quant models in TGI? it is a good solution for my use because it allows multiple simultaneous requests different from TGWEBUI, or is there any way to multiple requests in TGWEBUI? I have doubts about it