Error when launching
Hi MetalX,
I tried following your instructions and downloaded one of the safetensors along with all of the json and model files. When I launch using the oobabooga latest I get the following error:
FileNotFoundError: [Errno 2] No such file or directory: 'models\GPT4-X-Alpaca-30B-Int4\pytorch_model-00001-of-00061.bin'
It seems to be looking for pytorch .bin files. Do you have any suggestions about how I can fix that?
Thanks!
It appears those files are all listed within pytorch_model.bin.index.json. Can I generate those myself, using the safetensors?
use --wbits 4 with GPTQ
Works fine for me with this setup (change the folder name and keep the content same as screenshot)
I had the error, so I changed my command to this, which worked: python server.py --chat --model GPT4-X-Alpaca-30B-4bit --wbits 4
Edit: I also cloned https://huggingface.co/oobabooga/llama-tokenizer to the models folder