How to use with llama-cpp-python?
#1
by
lacoursj
- opened
Is there a way to make a .bin file from this git repo?
You don't use GPTQ models with llama-cpp-python, you use GGUF models with that.
And I have GGUF files for this model, here: https://huggingface.co/TheBloke/tulu-2-7B-GGUF
You can use one of those GGUF model files with llama-cpp-python.
PS. GGUF files usually have the extension .gguf now - .bin is old, from the early days of llama.cpp, back when it used GGML format. GGUF replaced GGML in August.
Thanks!