Inference API
#1
by
gsaivinay
- opened
Hello,
can we replace the previous pythia model with this latest iteration for the inference API widgets?
@sonatasv are you a member of the openassistant org? Thanks!
@julien-c Hello,
No, I'm not. just looking for default inference API to play with the model.
ok thanks! was just curious
@julien-c - on a completely different note, I saw you mentioning that you are running huggingface chat (https://huggingface.co/spaces/huggingchat/chat-ui) backend server models in AWS g5 instances. Would it be okay for you to share the server launch configuration parameters? I'm also using g5 instances with quantization, but generation speed is not as good as huggingface chat.
Huggingface chat is lightning fast.