Model doesn't call tools
#3
by infinitebibliography - opened
Anyone also has problems with this quantized model regarding tool calls? It simply stops responding (generating EOT token) before actually calling the tool. e.g.:
User: "Tell me what file test.txt is about"
LLM: "Sure, I will read the file."
It doesn't call the read tool or any other. Only when i continue a long conversation with already made tool calls, the model seems to understand it and is able to call tools.
I am using either opencode or openwebui, hosting the model with vllm as the documentation here states.
The model weights are proven working well for tool calls by Claude Code.