mlx-community

/

Mistral-7B-Instruct-v0.2

like 19

I am running this model using the mlx example code mistral.py, I downloaded the model, tokenizer and config files. When I run it, the model is loaded fine, but the gpu utilization is single digit, and the first 10 tokens output took like 5 mins. Using a M2 macbook air 16 GB. What is the issue?