INSTINSTINST
quantized version of the model spams INST.
Hey, I didn't really test gguf versions since the high score of the original model was kind of unexpected for me so I just prepared and published gguf version since people might ask it. Sorry to hear that though, which of the gguf version(s) is problematic? All or?
I have only tested Q4_K_M, but I think others have this problem too. For a couple of weeks now, models with this problem have been appearing constantly in the LLM leaderboard, and they are at the top of the list.
Yes, I can imagine my model inheriting this from other models since it's just a merge of a couple of top models. Sadly, top models in the leaderboard are sorted based on their higher precision capabilities by default, sadly no comparison for gguf versions available there...