Transformers
GGUF
llama
text-generation-inference

Nonsense output

#1
by FH501 - opened

Hello,

I appreciate your work a lot. However, I have tried the 4-bit GGUF models with llama.cpp and ctransformers and the result is always some nonsense. Do I need a specific version? What can I do to make them work properly?

Example
Input: Hello
Output: metro(%{{\ Stevens donner basically ls Init Usually Sep

Sign up or log in to comment