Gibberish output
#1
by
Adriato
- opened
I only get gibberish output in Oobabooga Text-gen from all Qwen-2 models. No other. Am I missing something?
if you're offloading to cuda, make sure to enable flash attention
Still gibberish with flash attention. Enabling no_offload_kqv did the trick, though. Using the Q6-K-quant. Thanks.