Quantization
#3
by
wrtn2
- opened
Hi, this is a great looking model! I tried quantizing it using llama.cpp's convert.py under WSL2 using:
python convert.py --outtype q8_0 --vocabtype bpe --outfile models/codellama-13b-oasst-sft-v10-8bit.gguf ../codellama-13b-oasst-sft-v10/
But it says vocab.json is missing:
FileNotFoundError: Could not find vocab.json
Any pointers appreciated.