Coherence at 16K
#2
by
SerialKicked
- opened
Using the Q6 GGUF (backend KCPP 1.64), the model doesn't have 16k context. It's coherent at 8K. It's usable at 12K (if a bit nonsensical), but anything above is just nonsense. Is that normal?
Yes, I noticed that too, unfortunately it's far from perfection/stability... I'm going to use a model with larger context to avoid any of this in the future. Thanks for the feedback.
Endevor
changed discussion status to
closed