Spaces:
Build error
How are you running this type of model on CPU? And why does it need OpenAI key?
Hello! During code research, I met some interesting moments, that I cannot understand. How are you running this type of model on CPU? And why does it need OpenAI key?
Hi @Nasa1423 ,
This space uses a modified version of https://github.com/huggingface/chat-ui
How are you running this type of model on CPU?
No, our model runs on GPU. The cpu only serves the frontend.
And why does it need OpenAI key?
No you don't need OpenAI key, I guess you mean https://github.com/huggingface/chat-ui#openai-api-compatible-models ?
I meant your Dockerfile, that requires env var with OpenAI key. And one more question, what GPU do you recommend for fast inference?
The OpenAI Key in the Dockerfile is for calling "the openai compatiable API" which backed/supported by our own GPU backend.
Two A800 or for 4090 is recommended for fast inference.