mlx-community

/

Qwen1.5-MoE-A2.7B-4bit

like 0

same here,

ModuleNotFoundError: No module named 'mlx_lm.models.qwen2_moe'

I'm still working on it.
I will update you guys when I'm done.

You can track the progress here: https://github.com/ml-explore/mlx-examples/pull/640

Done ✅

You can use it now, just update your mlx-lm and enjoy ;)

pip install -U mlx-lm

Btw, don’t use the base model with chat-template it produces bad results. Instead use it with the instruction tuned chat model here:

https://huggingface.co/mlx-community/Qwen1.5-MoE-A2.7B-Chat-4bit