TheBloke

/

Llama-2-7B-Chat-GGUF

like 375

Hi TheBloke!

I am havign trouble getting gguf to work with runpod, using your LLM template template. When I try to load with model loader llama.cpp, I get this error "TypeError: Llama.init() got an unexpected keyword argument ‘mul_mat_q’" and when I try to load with ctransformers, I get "ModuleNotFoundError: No module named ‘ctransformers’." Any thoughts on how to fix this would be much appreciated.