Still not ok with new llama-cpp version and llama.bin files
#5
by
Alwmd
- opened
I try all the 4Q bin file and always get the magic number error issue
Sorry for the long delay with this - I have finally uploaded GGUF models for this! Please use those instead; GGML is dead.
so GGML versions are dead, i think maybe you'll remove them from your account or something? I was getting errors due to it, just now got the GGUF working.
Have you positive experience on making it run on GPU with llama-cpp-python?
I haven't tried llama-cpp-python.
Just use https://huggingface.co/TheBloke/Llama-2-7B-GGUF and the other GGUF files. Yes I might remove these GGML models at some point, but there is already a clear note at the top of the repo indicating they are deprecated