122B-A10B 4bit

#35
by O-ishi - opened

Hi,

Running Qwen3.6-27B MLX on M4 Max 128GB via LM Studio. Looking to move to Qwen3.5-122B-A10B — the MoE activation ratio makes it a better fit for this hardware than the dense 27B.

The existing MLX conversions either drop the vision tower entirely or hit the transformers ≥5.4.0 / Qwen2VLImageProcessor PyTorch dependency issue in pure MLX environments.

You already solved the equivalent problem on Mistral-Medium-3.5-128B (mlx-vlm mistral3 sanitize bug, vision weights preserved at full precision). Any chance you'd apply the same treatment to Qwen3.5-122B-A10B?

Happy to test and report back.

Yep, since this is just a Jinja template, it works independently of model quantization or size. As long as your inference engine supports Jinja chat templates, you are good to go! Going to close this out.

froggeric changed discussion status to closed

Sign up or log in to comment