bartowski/internlm2_5-7b-chat-GGUF · Tool call tokens broken

FYI, the tool call tokens are broken with the current version of the models in this repo. Example:

<|im_start|>system
You are a helpful assistant.<|im_end|>
<|im_start|>system name=<|plugin|>
[{"name": "generate_image", "description": "Generates an image based on the given text prompt", "parameters": {"type": "object", "properties": {"prompt": {"type": "string", "description": "The text prompt used to guide image generation"}}, "required": ["prompt"]}}]<|im_end|>
<|im_start|>user
Draw a picture of a kitten.<|im_end|>
<|im_start|>assistant

I will call an image generation api to generate image[UNUSED_TOKEN_144][UNUSED_TOKEN_141]
{"name": "generate_image", "parameters": {"prompt": "A cute, fluffy kitten with big round eyes, sitting on a soft, warm blanket, looking up at the camera, natural lighting, high resolution, watercolor painting style, with a hint of magical realism."}}[UNUSED_TOKEN_143]
<|im_end|>[UNUSED_TOKEN_145]

I've posted corrected versions here.

Thanks for your ongoing work on quants. I use them all the time!