Hands down best 11B model I'd say, but I'm always guessing the context limit is around 4-5k
#2
by
bigfish3
- opened
Unless I'm wrong. I feel like the perplexity/common sense gets worse after 4k or something. Wish I knew the context limits of all of Sao's models, but either way really interesting.
Hmm Fimbulvetr is a solar based model, so 4k context. But rope scaling should work fine, I was able to handle 12k context fine with koboldcpp. Not sure about the values, kcpp does it automatically.
ohh. I see. Thanks. So most of them are 4k besides maybe the mixtrals.
I have had a lot of fun with this one, and really appreciate it!
Truthfully the only weakness of this model is the context limit. After about 8k, the quality dips for sure. But its still leaps better than Llama 3. I just wish it was as quick.