@stas on Hugging Face: "If you're trying to run MoE Mixtral-8x7b under DeepSpeed w/ HF Transformers…"

Join the community of Machine Learners and AI enthusiasts.

stas

posted an update Mar 12

Post

If you're trying to run MoE Mixtral-8x7b under DeepSpeed w/ HF Transformers it's likely to hang on the first forward.

The solution is here https://github.com/microsoft/DeepSpeed/pull/4966?_x_tr_sl=auto&_x_tr_tl=en&_x_tr_hl=en-US#issuecomment-1989671378

and you need deepspeed>=0.13.0

Thanks to Masahiro Tanaka for the fix.

In this post