Model tries too hard and goes wrong.
Had it for less then a houre and I see it go wrong every time.
Basic question answering it does perfectly.
But for example I integrated it into Hermes agent, it goes insane.
I see this happen in some models that are 'optimised for agentic use'. They are soo optimised they can't think straight anymore. Instead of going from point A to B, it wants to see the whole alphabet to know that A and B are right next to eachother, and surely not mentioned nowhere else.
Asking to read and summary a few txt files, for some reason it starts reading 3 different skills, does some python executions I think to find the files even tho I said where the files were.
To then find the files, and for some reason delegate and spin up agents to read and summarise these files.
After an houre of seeing these agents (it's own model) go back and forth for an houre I stopped it.
I ask it to find file A (yaml file) and change a value in that file.
it first starts to read every possible Skill that might have to do with that file, where 1 is relevant, the other the 5 others aren't relevant.
Then it finds the yaml file via the most insane search queries while the path is already mentioned in the skills he read.
When he finely finds it, it reads a part of the file.
For some reason it starts to read the '.py' files associated to the values it finds inside, not even relevant ones to the value I asked to change.
Then it does read a '.py' file, but the last 100 lines of a 2000+ line file. That is relevant to the value I asked to simply change.
Then it reads yet another '.py' file not relevant to the value I asked to change.
It somehow starts searching for other relevant files containing the value I asked to change in that 1 yaml file.
Reads another '.py' file, before then finely comming back to the right Yaml file.
And that is where I stopped it.
End result, this model might be good for some very specific and very guardrailed work. Cronjobs or something, but it is far from good for general agent use like in my case in Hermes.
you're not alone in this, I'm facing the same issues.
Similar behaviour for me. I was running it at Q4-K-S but without KV cache quantisation. Maybe this model is really sensitive to being quantized