Model produces garbage

#2
by WaveCut - opened

I'm using the Q4_1 quant variant, and regardless of what settings I've tried, I consistently receive the same output of endless @ until I stop the generation.

image.png

There is something wrong with it. How do I resolve this issue?

Crusoe AI org

Unfortunately, there are ongoing issues with llama3 tokenization in GGUF and I'm regenerating quants as potential fixes come in. Until then, I would recommend the mlx quants if you're on max otherwise exl2.

Sign up or log in to comment