Broken model?
Hi! Downloaded the Q8_0 version and checked the checksum, but this model seems broken (at least on koboldcpp-1.44.2) since it only outputs a single character (e. g. a colon) until the context is full.
It is a 16K model? What is the context extension technique this model uses? Maybe that's part that not correctly config-ed?
It's 16K according to the name. I tried it both with 4K and 16K context, same results in both cases. By trying different settings, I got it to output different characters, but every time it was just one that got repeated ad-infinitum. Deleted it afterwards and reported my issue here just in case others can confirm it or report which variant(s) work.
Maybe it was made with the buggy convert.py script that was updated 2 days ago
I feel like its broken as well:
USER: Tell me about Soil.
ASSISTANT: 1111111111111111111111111111111111111111111111111111111111111111111111
[the heck?]