@ucalyptus on Hugging Face: "PREM-1B-CHAT QUANTIZED INTO Q4 THEN SERVED IN WEBGPU DEMO OG model…"

Join the conversation

Join the community of Machine Learners and AI enthusiasts.

ucalyptus

posted an update about 1 month ago

Post

1430

PREM-1B-CHAT QUANTIZED INTO Q4
THEN SERVED IN WEBGPU DEMO

OG model premai-io/prem-1B-chat
Q4 model ucalyptus/prem-1B-chat-onnx-q4
WEBGPU demo ucalyptus/prem-1B-chat-webgpu

about 1 month ago

Outstanding issues :

Fix Q4 demo
https://huggingface.co/spaces/ucalyptus/prem-1B-chat-webgpu/discussions/1#664b621d8742922b9e4f3de8
Also work on fp16 (see what onnxruntime-web has to say about this)

In this post

ucalyptus Sayantan Das