The model is slow in my space.
#1
by
qitongwei
- opened
I deployed the model on this page：
“https://huggingface.co/spaces/qitongwei/touhouai-instruct-openllama-3b
Unfortunately, the model is slow and I have not seen it successfully output text.
I'm just an amateur, so I don't know what to do.😥
I sincerely ask for your help and really appreciate it.🙏
this is odd, you could run ggml/ctransformers for faster cpu inference (like in yoinked/mofu-webui)
Thanks, I will try it.🤗