how to use this model
#45
by
william0014
- opened
I have installed the FastChat .
then I use "python -m fastchat.serve.cli --model anon8231489123/vicuna-13b-GPTQ-4bit-128g". I got the error: Could not locate pytorch_model-00001-of-00003.bin inside anon8231489123/vicuna-13b-GPTQ-4bit-128g
it seems like your FastChat try to load a original model,however,this model is 4bit-128g.I think you should check the input params if there's a param like 4-bit
Try this
python -m fastchat.serve.cli --model-name anon8231489123/vicuna-13b-GPTQ-4bit-128g --wbits 4 --groupsize 128