Phi-3 support for GPU training with MPS on Mac

#52

by chaishirin - opened May 2

May 2

Hi.

Would first like to thank you for this awesome SLM for open-sourcing it. I have tested it on my RTX3060 without any bottle-necks. Now when I try it on my Mac M1 chip with MPS backend the outputs are weird and incoherent. Using CPU on the Mac generates correct/coherent output.

Saw that modeling_phi3.py falls back to gpu if not mps else it uses cpu. Do you know what the incoherent output could stem from?

gugarosa

Microsoft org May 2

Would running Phi-3 with MLX instead of MPS be better for an M1 chip?

nguyenbh changed discussion status to closed Jul 1

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment