Hugging Face's logo

rustformers
/
mpt-7b-ggml

Text Generation
Transformers
English
mpt
llm-rs
ggml
text-generation-inference
Model card Files Community
5

Stuck When Running Python Inference as Described in Model Card?

#4
by fffiend - opened
Discussion
fffiend

I basically get stuck on :
Fetching 1 files: 100%|█████████████████████████████████| 1/1 [00:00<00:00, 1.55it/s]

And then nothing happens. I'm running a Macbook Pro with an M2 chip and 8GB RAM.

fffiend

Resolved! It was because I didn't have enough CPU mem available for it to generate a completion :D

fffiend changed discussion status to closed
LLukas22
 rustformers org

@fffiend Yeah, 8GB RAM isn't that much, i would recommend to run the q4_0 versions of the models, those should fit. Or maybe try different models like redpajama-3B which should fit more easily into your available RAM.

Sign up or log in to comment