TheBloke/CodeLlama-34B-Instruct-GGUF only generates "text": "\n\n\n\n\n\n\n\n

#4
by hiteshsom - opened

Hi I am using Langchain agents and in that I am using TheBloke/CodeLlama-34B-Instruct-GGUF llm. But it only gives output
"text": "\n\n\n\n\n\n\n\n

However mistral-7b-instruct-v0.1.Q6_K.gguf worked fine.

Environment:
CPU
RAM: 32 GB
Windows

Had the same problem, mostly it repeated newline characters, but sometimes it started to repead whole sentences of the response.
Also it went into an infinite loop (or i did not have enough patience and killed the process early).

Fixed it by adjusting my ropeconfig:
localAI yaml style:
rope_freq_base: 1000000
rope_freq_scale: 1.0

Original comment that lead me to the fix:
https://huggingface.co/TheBloke/Phind-CodeLlama-34B-v2-GGUF/discussions/2

Sign up or log in to comment