Llama-3-8b-Ultra-Instruct-GGUF not working properly
#1
by
JDE65
- opened
Hello,
I tested Phi-3-mini, mistral-7b-Instruct-GGUF and Llama-3-8b-Ultra-Instruct-GGUF in Q6_K versions with llama.cpp on a model with a RAG and I get troubles only with Llama3 with inconsistent sentences.
I then tested another release of Llama-3-8b-Instruct-GGUF from QuantFactory and from Nous, also looking for Q4 and Q8 quantized versions with llama.cpp and, again, Llama3 was systematically delivering inconsistent sentences while the others (Phi3 and Mistral) where performing well.
Any idea why and how to solve it ?
THX in advance