Repo lacks config.json
#2
by
Xabab
- opened
Ditto. Can't use out of the box with aphrodite engine
Hi for the latest aphrodite release, you need to convert the gguf to pytorch state_dict before loading. This shouldn't be required by the next release (or if you build from source).
Use this script to convert. e.g.:
python gguf_to_torch.py --input Clover3-17B.q4_k_s.gguf --output ./clover-gguf --safetensors