Join the conversation

Join the community of Machine Learners and AI enthusiasts.

Sign Up
ivanfioravanti 
posted an update 14 days ago
Post
Repo with scripts to create your own moe models using Apple mlx is constantly updated by @mzbac here: https://github.com/mzbac/mlx-moe

It's an amazing resource to learn inner workings of lora on moe with mlx.
It uses WizardLM/WizardLM_evol_instruct_70k as default dataset, but can be easily tweak to use any model or dataset fro HF

Have fun with it!

thanks a lot for sharing 🤗 also cc @reach-vb