Someone please make GGUF versions (-)
#5
by
huggingfacess
- opened
Someone please make GGUF versions
The correct way.
python3 llama.cpp/convert-hf-to-gguf.py GemMoE-Beta-1/ --outfile ggml-model-f16.gguf --outtype f16
don't use this.
python3 llama.cpp/convert.py
I will make one - but I am currently updating the model to version 1.1. There was a decently large bug in this one that caused vram use to go out of control.