mlx-community

/

Muse-Glimmer-30B-4bit

like 11

Follow

MLX Community 15.9k

Use Unsloth Studio , It Supports Tools And It Works For Me

Yeah, leverage Unsloth, they really have awesome tools, recommended

@kolyno25 agreed! so many of these MLX conversions remove the stuff that makes the models useful and what they're advertised as being good at

@siddh07ETH this model is missing tool use in unsloth too. what are you seeing?

Hey Everything Works Fine On My Side , Please Take A Look At The Document It May Useful

https://unsloth.ai/docs/models/muse-glimmer

Thanks

@Siddh07ETH we are all talking about MLX flavor. That's about GGUF... no?

Yeah Sorry Mate Noted That , I Think MLX Template Does Not Using Tool Call As A Default , For Now Use GGUF , Once I Found Solution on my side i will let you know mate

the mlx conversion is not stripping tool support. i checked it properly rather than eyeballing the files.

pulled chat_template.jinja from meta's repo and from this one, rendered both through the same jinja sandbox with the same messages and the same tool definition, and the output is byte identical, 2083 characters on both sides, atem:function_calls protocol and the tool name intact.

the two files look wildly different if you diff them, which i think is where the suspicion comes from. the original is 211 lines and this one is 1 line, because the mlx side is minified. what got dropped is the jinja comment header and the readability indentation, and every tag in that template already uses the {%- -%} whitespace stripping markers so none of that ever reached the rendered output anyway. both files contain the same 13 tools references and the same ATEM macros.

also checked the layout in case something reads the template from the wrong place. both repos ship it as chat_template.jinja with no chat_template key inside tokenizer_config.json, and meta's original does exactly the same, so nothing is out of place there either.

so whatever is going on is downstream in the lm studio mlx engine, not in the quant. probably worth filing on the lmstudio bug tracker rather than here, the quant repos cannot fix it.

@jhmonroe on the wider point, at least for this model the conversion is faithful, i could not find anything removed.

@divinetribe thanks for such a detailed and thoughtful explanation!

so you're saying it's just LM Studio showing it has no tool use even though it still does, right?

opened ticket with them: https://github.com/lmstudio-ai/lmstudio-bug-tracker/issues/2273

thanks!

@jhmonroe yeah, that's what i'm saying, with one thing i want to be precise about. what i verified is that this quant isn't what removes tool support, the template that carries it is byte identical to meta's. i didn't verify that a tool call actually round trips through lm studio's mlx engine, so all i can tell you is nothing is missing on the model side for it to work.

i'm pulling this exact 4bit down now and running it through the mlx server with a real tool definition, outside lm studio. if the model emits the atem call there, that pins it on their runtime and your ticket gets a clean datapoint. i'll post what i find here either way.

thanks for filing it.

@divinetribe thank you! team work makes the dream work. love OSS community <3

ran it. same 4bit repo, stock mlx-vlm 0.6.12 server on an m5, no lm studio in the loop. one tool defined, temperature 0, stream off.

came back with finish_reason: tool_calls and {"name": "get_weather", "arguments": "{\"city\": \"Denver\"}"}, content empty, which is what a pure tool turn should look like. so the quant emits tool calls fine and the atem protocol survives the conversion intact. that puts it in lm studio's mlx engine, feel free to link this on ticket 2273.

one unrelated thing i noticed while in there, on a plain prompt the reasoning channel comes back inside content instead of split out, raw text starts with to=self<|message|> and the real answer only shows after <|eom|>. that's a separate known issue being worked on upstream, not your tool problem.