4k context by default?
#1
by
mclassHF2023
- opened
I know the model supports 128k context, and your exl2 version also shows that by default. But the GGUF version shows 4k context by default.
My question is: is it safe to simply set this context higher, or do I need to modify something? Also: is this intentional?
It isn't intentional, that's interesting, I think you may need to adjust rope values to what it displays in the config.json of the main model